mirror of
https://github.com/openclaw/clawhub.git
synced 2026-08-14 08:52:21 +00:00
Compare commits
235
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
f7c72bb56c | ||
|
|
4787be4eb1 | ||
|
|
89246f1927 | ||
|
|
f4ddccbead | ||
|
|
3cafcbf873 | ||
|
|
13064a7897 | ||
|
|
194c22f4dd | ||
|
|
a693b945fa | ||
|
|
9bef672541 | ||
|
|
9551cac37b | ||
|
|
eb4138fbb3 | ||
|
|
5fbead624b | ||
|
|
35094177e6 | ||
|
|
faa5c9f2b5 | ||
|
|
c3314c2d01 | ||
|
|
7dfa19157c | ||
|
|
44acf86ac1 | ||
|
|
a0ebc1b50a | ||
|
|
88dbb69a23 | ||
|
|
df9acd27e4 | ||
|
|
dbd5d4042c | ||
|
|
ebe82b7e18 | ||
|
|
4c566268a9 | ||
|
|
e54fc1939a | ||
|
|
530e39eedc | ||
|
|
8b87c31a99 | ||
|
|
f7bc8b6349 | ||
|
|
5b8f09167a | ||
|
|
17fbd13bc9 | ||
|
|
aab7dc9ba4 | ||
|
|
dde8796790 | ||
|
|
acc6d292de | ||
|
|
2236ed7be1 | ||
|
|
f6fb7ccfc0 | ||
|
|
b73758c7c8 | ||
|
|
fbc07c5617 | ||
|
|
80e5aec577 | ||
|
|
f869b31ad6 | ||
|
|
9a853f2fcc | ||
|
|
b4a7540157 | ||
|
|
0ea1127a2b | ||
|
|
aeab23a6d6 | ||
|
|
15bc4440cc | ||
|
|
11a20f5755 | ||
|
|
05f8674628 | ||
|
|
d2b2252770 | ||
|
|
a2387253ec | ||
|
|
411260767b | ||
|
|
731d0ce0c5 | ||
|
|
361f2affde | ||
|
|
a17f7bb07e | ||
|
|
7980788ea6 | ||
|
|
2d03b827d3 | ||
|
|
835094ea2c | ||
|
|
7bd7e4c99e | ||
|
|
1722a48055 | ||
|
|
29178898bb | ||
|
|
b16861f422 | ||
|
|
6896e61fa1 | ||
|
|
cf137aa592 | ||
|
|
a28d014d4f | ||
|
|
b2038fc931 | ||
|
|
4a72b543b2 | ||
|
|
54e99c8cc2 | ||
|
|
5826001795 | ||
|
|
0708a43fde | ||
|
|
9a45c371fc | ||
|
|
311a123fbe | ||
|
|
59e93862ed | ||
|
|
8fcd53f899 | ||
|
|
ba2c73e180 | ||
|
|
298cbdd6db | ||
|
|
fa87dc3509 | ||
|
|
f636b31fca | ||
|
|
b255b5865f | ||
|
|
5003c1bec8 | ||
|
|
a8a6242f87 | ||
|
|
39c0fa2531 | ||
|
|
9df6fe37c5 | ||
|
|
d5d806516d | ||
|
|
67c74e10ea | ||
|
|
3501af6bb4 | ||
|
|
051b1dafcd | ||
|
|
383844cacf | ||
|
|
b7923edbfd | ||
|
|
655c914c77 | ||
|
|
b15eeab93f | ||
|
|
a39f07427e | ||
|
|
1a42207879 | ||
|
|
9bd3a63edf | ||
|
|
be4c51ed1b | ||
|
|
8711de4441 | ||
|
|
f4db0ee32b | ||
|
|
f1cf715b89 | ||
|
|
57ed1f62a4 | ||
|
|
bde7529fbe | ||
|
|
5892718a23 | ||
|
|
922eecdbd9 | ||
|
|
e53a4433ae | ||
|
|
68870a1dcc | ||
|
|
c6aaf27886 | ||
|
|
f71db92fc3 | ||
|
|
07a1da4285 | ||
|
|
cb9d854e92 | ||
|
|
74ec5191e0 | ||
|
|
cf98936105 | ||
|
|
dd9c42be28 | ||
|
|
e8a4e094e7 | ||
|
|
068dd78a05 | ||
|
|
2f5e17f56e | ||
|
|
5302bf8598 | ||
|
|
749c89e77b | ||
|
|
fd08d74071 | ||
|
|
67c8e188e3 | ||
|
|
bad07ff95c | ||
|
|
a7547eee51 | ||
|
|
fddbb35b40 | ||
|
|
ba313e2d0a | ||
|
|
f6f31ac78d | ||
|
|
f32500b1b4 | ||
|
|
5bc6402b5e | ||
|
|
9721461aca | ||
|
|
91f80c264a | ||
|
|
712302eb30 | ||
|
|
f044164bae | ||
|
|
997da8857a | ||
|
|
1a31e07332 | ||
|
|
cb761a73a9 | ||
|
|
d4ad4ea489 | ||
|
|
d7c9126c4b | ||
|
|
5dc6720d9e | ||
|
|
087f2c75b0 | ||
|
|
03c42fa947 | ||
|
|
572110b1d5 | ||
|
|
812276f1b7 | ||
|
|
b9676f674c | ||
|
|
32d601aab7 | ||
|
|
4aae925e76 | ||
|
|
e41fd677bb | ||
|
|
31520ef02a | ||
|
|
e829cb1ae6 | ||
|
|
24cc59a425 | ||
|
|
522fa22026 | ||
|
|
7e76336ae4 | ||
|
|
e41d5d6314 | ||
|
|
c62ab8bc96 | ||
|
|
6df3a3b9ef | ||
|
|
7452cf6f69 | ||
|
|
8ded548da2 | ||
|
|
4ff2d1a0cd | ||
|
|
b4e8a26eb4 | ||
|
|
7ff601bcb8 | ||
|
|
70af109cb2 | ||
|
|
1504708208 | ||
|
|
4af2bd50a7 | ||
|
|
88fe310ff1 | ||
|
|
0a31b31f1b | ||
|
|
079e390da6 | ||
|
|
15da02c70a | ||
|
|
8b289618ba | ||
|
|
4ae0406948 | ||
|
|
78d0a637fa | ||
|
|
65bc5d3335 | ||
|
|
8592272720 | ||
|
|
15f5769cda | ||
|
|
bb2a05e501 | ||
|
|
af29ca7f9d | ||
|
|
13caea7cfd | ||
|
|
86259eef42 | ||
|
|
bf2ecab92e | ||
|
|
70516bd0f5 | ||
|
|
66fca9a286 | ||
|
|
a03fe3bbd0 | ||
|
|
d3d9298389 | ||
|
|
2fd0aa01d5 | ||
|
|
5aad36dab0 | ||
|
|
1bab92313f | ||
|
|
f77993d614 | ||
|
|
3a28344c53 | ||
|
|
ea49143fe0 | ||
|
|
bd38fef5c5 | ||
|
|
b7a015523d | ||
|
|
d36f98faa1 | ||
|
|
eeb0ecd932 | ||
|
|
51beceeb20 | ||
|
|
a8a687eba2 | ||
|
|
b9341202c7 | ||
|
|
d701c2a33b | ||
|
|
79d17a91ed | ||
|
|
2f15202a68 | ||
|
|
342a2b1ca4 | ||
|
|
75915cd2b5 | ||
|
|
5395d9d159 | ||
|
|
87c236e8ce | ||
|
|
d8db9b99a2 | ||
|
|
d8e1f0daa1 | ||
|
|
8a350d953c | ||
|
|
3fb3150ee2 | ||
|
|
7dbc0fc3bb | ||
|
|
fc4f8644eb | ||
|
|
dda6d55fbf | ||
|
|
e014759b40 | ||
|
|
2186c41c48 | ||
|
|
c30c182478 | ||
|
|
3080567964 | ||
|
|
f304541561 | ||
|
|
91224ada13 | ||
|
|
d5fbaeef81 | ||
|
|
3706018b72 | ||
|
|
62e616f635 | ||
|
|
9013d324c8 | ||
|
|
c1363ec8d0 | ||
|
|
7e09196f92 | ||
|
|
807043b4b0 | ||
|
|
972fe35935 | ||
|
|
16ee540f5d | ||
|
|
f541882d55 | ||
|
|
95bc156747 | ||
|
|
bf7422022f | ||
|
|
230e5b91f8 | ||
|
|
70dcf21e37 | ||
|
|
932a1fb30c | ||
|
|
44fe60b701 | ||
|
|
1c057ca9b9 | ||
|
|
9f793d1336 | ||
|
|
aa9295bea9 | ||
|
|
370eea4977 | ||
|
|
59e6819020 | ||
|
|
52b633f4e9 | ||
|
|
6b1e6ca1c9 | ||
|
|
6c112eccb7 | ||
|
|
f9e9effcdd | ||
|
|
b5d2d0fefa | ||
|
|
48d0fc91f3 | ||
|
|
fefb2340a8 |
@@ -27,15 +27,22 @@ jobs:
|
||||
|
||||
- name: Test
|
||||
run: bun run test
|
||||
env:
|
||||
VITE_CONVEX_URL: https://example.invalid
|
||||
|
||||
- name: Coverage
|
||||
run: bun run coverage
|
||||
env:
|
||||
VITE_CONVEX_URL: https://example.invalid
|
||||
|
||||
- name: ClawHub CLI Verify
|
||||
run: bun run --cwd packages/clawhub verify
|
||||
|
||||
- name: Typecheck
|
||||
run: |
|
||||
bunx tsc --noEmit
|
||||
bunx tsc -p packages/schema/tsconfig.json --noEmit
|
||||
bunx tsc -p packages/clawdhub/tsconfig.json --noEmit
|
||||
bunx tsc -p packages/clawhub/tsconfig.json --noEmit
|
||||
|
||||
- name: Build
|
||||
run: bun run build
|
||||
|
||||
@@ -0,0 +1,314 @@
|
||||
name: ClawHub CLI NPM Release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
tag:
|
||||
description: Release tag to publish, for example v0.10.0
|
||||
required: true
|
||||
type: string
|
||||
preflight_only:
|
||||
description: Run validation/build only and skip the gated publish job
|
||||
required: true
|
||||
default: false
|
||||
type: boolean
|
||||
preflight_run_id:
|
||||
description: Existing successful preflight workflow run id to promote without rebuilding
|
||||
required: false
|
||||
type: string
|
||||
|
||||
concurrency:
|
||||
group: clawhub-cli-npm-release-${{ inputs.tag }}
|
||||
cancel-in-progress: false
|
||||
|
||||
env:
|
||||
FORCE_JAVASCRIPT_ACTIONS_TO_NODE24: "true"
|
||||
NODE_VERSION: "24.x"
|
||||
BUN_VERSION: "1.3.10"
|
||||
|
||||
jobs:
|
||||
preflight_clawhub_cli_npm:
|
||||
if: ${{ inputs.preflight_only }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Forbid preflight artifact promotion on validation-only runs
|
||||
if: ${{ inputs.preflight_run_id != '' }}
|
||||
run: |
|
||||
echo "preflight_run_id is only valid for real publish runs."
|
||||
exit 1
|
||||
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: refs/tags/${{ inputs.tag }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
registry-url: https://registry.npmjs.org
|
||||
|
||||
- name: Setup Bun
|
||||
uses: oven-sh/setup-bun@e3914758a49697077f7bcd190d36582a61667aad
|
||||
with:
|
||||
bun-version: ${{ env.BUN_VERSION }}
|
||||
|
||||
- name: Install dependencies
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
- name: Resolve CLI package directory
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [[ -d "packages/clawhub" ]]; then
|
||||
echo "PACKAGE_DIR=packages/clawhub" >> "$GITHUB_ENV"
|
||||
elif [[ -d "packages/clawdhub" ]]; then
|
||||
echo "PACKAGE_DIR=packages/clawdhub" >> "$GITHUB_ENV"
|
||||
else
|
||||
echo "Unable to find clawhub CLI package directory." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Ensure version is not already published
|
||||
env:
|
||||
PREFLIGHT_ONLY: ${{ inputs.preflight_only }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
|
||||
process.stdout.write(String(pkg.version ?? "").trim());
|
||||
EOF
|
||||
)"
|
||||
|
||||
if npm view "clawhub@${PACKAGE_VERSION}" version >/dev/null 2>&1; then
|
||||
if [[ "${PREFLIGHT_ONLY}" == "true" ]]; then
|
||||
echo "clawhub@${PACKAGE_VERSION} is already published on npm; continuing because preflight_only=true."
|
||||
exit 0
|
||||
fi
|
||||
echo "clawhub@${PACKAGE_VERSION} is already published on npm."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Publishing clawhub@${PACKAGE_VERSION}"
|
||||
|
||||
- name: Validate release tag and package metadata
|
||||
env:
|
||||
RELEASE_TAG: ${{ inputs.tag }}
|
||||
RELEASE_MAIN_REF: origin/main
|
||||
run: |
|
||||
set -euo pipefail
|
||||
RELEASE_SHA="$(git rev-parse HEAD)"
|
||||
export RELEASE_SHA
|
||||
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
|
||||
node scripts/clawhub-cli-npm-release-check.mjs
|
||||
|
||||
- name: Verify CLI package
|
||||
run: bun run --cwd "$PACKAGE_DIR" verify
|
||||
|
||||
- name: Pack prepared npm tarball
|
||||
id: packed_tarball
|
||||
env:
|
||||
RELEASE_TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
pushd "$PACKAGE_DIR" >/dev/null
|
||||
PACK_JSON="$(npm pack --json --ignore-scripts)"
|
||||
echo "$PACK_JSON"
|
||||
PACK_PATH="$(printf '%s\n' "$PACK_JSON" | node --input-type=module -e 'const chunks=[]; process.stdin.on("data", (chunk) => chunks.push(chunk)); process.stdin.on("end", () => { const parsed = JSON.parse(Buffer.concat(chunks).toString("utf8")); const first = Array.isArray(parsed) ? parsed[0] : null; if (!first || typeof first.filename !== "string" || !first.filename) process.exit(1); process.stdout.write(first.filename); });')"
|
||||
popd >/dev/null
|
||||
if [[ -z "${PACK_PATH}" || ! -f "${PACKAGE_DIR}/${PACK_PATH}" ]]; then
|
||||
echo "npm pack did not produce a tarball file." >&2
|
||||
exit 1
|
||||
fi
|
||||
RELEASE_SHA="$(git rev-parse HEAD)"
|
||||
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
|
||||
process.stdout.write(String(pkg.version ?? "").trim());
|
||||
EOF
|
||||
)"
|
||||
ARTIFACT_DIR="$RUNNER_TEMP/clawhub-cli-npm-preflight"
|
||||
rm -rf "$ARTIFACT_DIR"
|
||||
mkdir -p "$ARTIFACT_DIR"
|
||||
cp "${PACKAGE_DIR}/${PACK_PATH}" "$ARTIFACT_DIR/"
|
||||
printf '%s\n' "$RELEASE_TAG" > "$ARTIFACT_DIR/release-tag.txt"
|
||||
printf '%s\n' "$RELEASE_SHA" > "$ARTIFACT_DIR/release-sha.txt"
|
||||
printf '%s\n' "$PACKAGE_VERSION" > "$ARTIFACT_DIR/package-version.txt"
|
||||
echo "dir=$ARTIFACT_DIR" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Upload prepared npm publish bundle
|
||||
uses: actions/upload-artifact@v7
|
||||
with:
|
||||
name: clawhub-cli-npm-preflight-${{ inputs.tag }}
|
||||
path: ${{ steps.packed_tarball.outputs.dir }}
|
||||
if-no-files-found: error
|
||||
|
||||
validate_publish_request:
|
||||
if: ${{ !inputs.preflight_only }}
|
||||
runs-on: ubuntu-latest
|
||||
permissions:
|
||||
contents: read
|
||||
steps:
|
||||
- name: Require main workflow ref for publish
|
||||
env:
|
||||
WORKFLOW_REF: ${{ github.ref }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [[ "${WORKFLOW_REF}" != "refs/heads/main" ]]; then
|
||||
echo "Real publish runs must be dispatched from main. Use preflight_only=true for branch validation."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Require preflight artifact promotion on real publish
|
||||
env:
|
||||
PREFLIGHT_RUN_ID: ${{ inputs.preflight_run_id }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [[ -z "${PREFLIGHT_RUN_ID}" ]]; then
|
||||
echo "Real publish requires preflight_run_id from a successful npm preflight run." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
publish_clawhub_cli_npm:
|
||||
needs: [validate_publish_request]
|
||||
if: ${{ !inputs.preflight_only }}
|
||||
runs-on: ubuntu-latest
|
||||
environment: npm-release
|
||||
permissions:
|
||||
actions: read
|
||||
contents: read
|
||||
id-token: write
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v6
|
||||
with:
|
||||
ref: refs/tags/${{ inputs.tag }}
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Setup Node
|
||||
uses: actions/setup-node@v6
|
||||
with:
|
||||
node-version: ${{ env.NODE_VERSION }}
|
||||
registry-url: https://registry.npmjs.org
|
||||
|
||||
- name: Resolve CLI package directory
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [[ -d "packages/clawhub" ]]; then
|
||||
echo "PACKAGE_DIR=packages/clawhub" >> "$GITHUB_ENV"
|
||||
elif [[ -d "packages/clawdhub" ]]; then
|
||||
echo "PACKAGE_DIR=packages/clawdhub" >> "$GITHUB_ENV"
|
||||
else
|
||||
echo "Unable to find clawhub CLI package directory." >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Ensure version is not already published
|
||||
run: |
|
||||
set -euo pipefail
|
||||
PACKAGE_VERSION="$(node --input-type=module <<'EOF'
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
|
||||
process.stdout.write(String(pkg.version ?? "").trim());
|
||||
EOF
|
||||
)"
|
||||
|
||||
if npm view "clawhub@${PACKAGE_VERSION}" version >/dev/null 2>&1; then
|
||||
echo "clawhub@${PACKAGE_VERSION} is already published on npm."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Publishing clawhub@${PACKAGE_VERSION}"
|
||||
|
||||
- name: Verify preflight run metadata
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
PREFLIGHT_RUN_ID: ${{ inputs.preflight_run_id }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
RUN_JSON="$(gh run view "$PREFLIGHT_RUN_ID" --repo "$GITHUB_REPOSITORY" --json workflowName,headBranch,event,conclusion,url)"
|
||||
printf '%s' "$RUN_JSON" | node --input-type=module -e 'const chunks=[]; process.stdin.on("data", (chunk) => chunks.push(chunk)); process.stdin.on("end", () => { const run = JSON.parse(Buffer.concat(chunks).toString("utf8")); const checks = [["workflowName", "ClawHub CLI NPM Release"], ["headBranch", "main"], ["event", "workflow_dispatch"], ["conclusion", "success"]]; for (const [key, expected] of checks) { if (run[key] !== expected) { console.error(`Referenced npm preflight run ${process.env.PREFLIGHT_RUN_ID} must have ${key}=${expected}, got ${run[key] ?? "<missing>"}.`); process.exit(1); } } console.log(`Using npm preflight run ${process.env.PREFLIGHT_RUN_ID}: ${run.url}`); });'
|
||||
|
||||
- name: Download prepared npm tarball
|
||||
uses: actions/download-artifact@v8
|
||||
with:
|
||||
name: clawhub-cli-npm-preflight-${{ inputs.tag }}
|
||||
path: preflight-tarball
|
||||
repository: ${{ github.repository }}
|
||||
run-id: ${{ inputs.preflight_run_id }}
|
||||
github-token: ${{ github.token }}
|
||||
|
||||
- name: Validate release tag and package metadata
|
||||
env:
|
||||
RELEASE_TAG: ${{ inputs.tag }}
|
||||
RELEASE_MAIN_REF: origin/main
|
||||
run: |
|
||||
set -euo pipefail
|
||||
RELEASE_SHA="$(git rev-parse HEAD)"
|
||||
export RELEASE_SHA
|
||||
git fetch --no-tags origin +refs/heads/main:refs/remotes/origin/main
|
||||
node scripts/clawhub-cli-npm-release-check.mjs
|
||||
|
||||
- name: Verify prepared tarball provenance
|
||||
env:
|
||||
RELEASE_TAG: ${{ inputs.tag }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
EXPECTED_RELEASE_SHA="$(git rev-parse HEAD)"
|
||||
EXPECTED_PACKAGE_VERSION="$(node --input-type=module <<'EOF'
|
||||
import { readFileSync } from "node:fs";
|
||||
|
||||
const pkg = JSON.parse(readFileSync(`./${process.env.PACKAGE_DIR}/package.json`, "utf8"));
|
||||
process.stdout.write(String(pkg.version ?? "").trim());
|
||||
EOF
|
||||
)"
|
||||
TAG_FILE="preflight-tarball/release-tag.txt"
|
||||
SHA_FILE="preflight-tarball/release-sha.txt"
|
||||
VERSION_FILE="preflight-tarball/package-version.txt"
|
||||
if [[ ! -f "$TAG_FILE" || ! -f "$SHA_FILE" || ! -f "$VERSION_FILE" ]]; then
|
||||
echo "Prepared preflight metadata is missing." >&2
|
||||
ls -la preflight-tarball >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
ARTIFACT_RELEASE_TAG="$(tr -d '\r\n' < "$TAG_FILE")"
|
||||
ARTIFACT_RELEASE_SHA="$(tr -d '\r\n' < "$SHA_FILE")"
|
||||
ARTIFACT_PACKAGE_VERSION="$(tr -d '\r\n' < "$VERSION_FILE")"
|
||||
if [[ "$ARTIFACT_RELEASE_TAG" != "$RELEASE_TAG" ]]; then
|
||||
echo "Prepared preflight tag mismatch: expected $RELEASE_TAG, got $ARTIFACT_RELEASE_TAG" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$ARTIFACT_RELEASE_SHA" != "$EXPECTED_RELEASE_SHA" ]]; then
|
||||
echo "Prepared preflight SHA mismatch: expected $EXPECTED_RELEASE_SHA, got $ARTIFACT_RELEASE_SHA" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [[ "$ARTIFACT_PACKAGE_VERSION" != "$EXPECTED_PACKAGE_VERSION" ]]; then
|
||||
echo "Prepared preflight package version mismatch: expected $EXPECTED_PACKAGE_VERSION, got $ARTIFACT_PACKAGE_VERSION" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Resolve publish tarball
|
||||
id: publish_tarball
|
||||
run: |
|
||||
set -euo pipefail
|
||||
TARBALL_PATH="$(find preflight-tarball -type f -name '*.tgz' -print | sort | tail -n 1)"
|
||||
if [[ -z "$TARBALL_PATH" ]]; then
|
||||
echo "Prepared preflight tarball not found." >&2
|
||||
ls -la preflight-tarball >&2 || true
|
||||
exit 1
|
||||
fi
|
||||
echo "path=$TARBALL_PATH" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Publish
|
||||
run: |
|
||||
set -euo pipefail
|
||||
publish_target="${{ steps.publish_tarball.outputs.path }}"
|
||||
if [[ -n "${publish_target}" ]]; then
|
||||
publish_target="./${publish_target}"
|
||||
fi
|
||||
bash scripts/clawhub-cli-npm-publish.sh --publish "${publish_target}"
|
||||
@@ -1,46 +1,100 @@
|
||||
name: Deploy
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
inputs:
|
||||
target:
|
||||
description: "What to deploy"
|
||||
required: true
|
||||
default: full
|
||||
type: choice
|
||||
options:
|
||||
- full
|
||||
- backend
|
||||
- frontend
|
||||
|
||||
concurrency:
|
||||
group: deploy-production
|
||||
cancel-in-progress: true
|
||||
|
||||
jobs:
|
||||
preflight-secrets:
|
||||
validate-deploy-request:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 5
|
||||
outputs:
|
||||
deploy_backend: ${{ steps.mode.outputs.deploy_backend }}
|
||||
deploy_frontend: ${{ steps.mode.outputs.deploy_frontend }}
|
||||
run_smoke: ${{ steps.mode.outputs.run_smoke }}
|
||||
target: ${{ steps.mode.outputs.target }}
|
||||
steps:
|
||||
- name: Require main ref for production deploy
|
||||
run: |
|
||||
set -euo pipefail
|
||||
if [[ "${GITHUB_REF}" != "refs/heads/main" ]]; then
|
||||
echo "Production deploys must run from main."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Resolve deploy mode
|
||||
id: mode
|
||||
run: |
|
||||
set -euo pipefail
|
||||
target="${{ inputs.target }}"
|
||||
case "$target" in
|
||||
full)
|
||||
echo "deploy_backend=true" >> "$GITHUB_OUTPUT"
|
||||
echo "deploy_frontend=true" >> "$GITHUB_OUTPUT"
|
||||
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
|
||||
;;
|
||||
backend)
|
||||
echo "deploy_backend=true" >> "$GITHUB_OUTPUT"
|
||||
echo "deploy_frontend=false" >> "$GITHUB_OUTPUT"
|
||||
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
|
||||
;;
|
||||
frontend)
|
||||
echo "deploy_backend=false" >> "$GITHUB_OUTPUT"
|
||||
echo "deploy_frontend=true" >> "$GITHUB_OUTPUT"
|
||||
echo "run_smoke=true" >> "$GITHUB_OUTPUT"
|
||||
;;
|
||||
*)
|
||||
echo "Unsupported deploy target: $target" >&2
|
||||
exit 1
|
||||
;;
|
||||
esac
|
||||
echo "target=$target" >> "$GITHUB_OUTPUT"
|
||||
|
||||
deploy-production:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 45
|
||||
needs: validate-deploy-request
|
||||
environment:
|
||||
name: Production
|
||||
url: https://clawhub.ai
|
||||
env:
|
||||
CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY }}
|
||||
PLAYWRIGHT_AUTH_STORAGE_STATE_JSON: ${{ secrets.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON }}
|
||||
PLAYWRIGHT_BASE_URL: https://clawhub.ai
|
||||
steps:
|
||||
- name: Check deploy secrets
|
||||
- name: Check deploy configuration
|
||||
run: |
|
||||
set -euo pipefail
|
||||
missing=()
|
||||
|
||||
if [[ -z "$CONVEX_DEPLOY_KEY" ]]; then
|
||||
if [[ "${{ needs.validate-deploy-request.outputs.deploy_backend }}" == "true" && -z "$CONVEX_DEPLOY_KEY" ]]; then
|
||||
missing+=("CONVEX_DEPLOY_KEY")
|
||||
fi
|
||||
|
||||
if (( ${#missing[@]} > 0 )); then
|
||||
echo "::error::Missing required GitHub Actions secrets: ${missing[*]}"
|
||||
echo "::error::Missing required production environment secrets: ${missing[*]}"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "Deploy target: ${{ needs.validate-deploy-request.outputs.target }}"
|
||||
|
||||
if [[ -z "$PLAYWRIGHT_AUTH_STORAGE_STATE_JSON" ]]; then
|
||||
echo "PLAYWRIGHT_AUTH_STORAGE_STATE_JSON not set; authenticated smoke will be skipped."
|
||||
fi
|
||||
|
||||
deploy-convex:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
needs: preflight-secrets
|
||||
env:
|
||||
CONVEX_DEPLOY_KEY: ${{ secrets.CONVEX_DEPLOY_KEY }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: oven-sh/setup-bun@e3914758a49697077f7bcd190d36582a61667aad
|
||||
@@ -51,35 +105,38 @@ jobs:
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
- name: Stamp Convex build SHA
|
||||
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
|
||||
run: bunx convex env set APP_BUILD_SHA "${GITHUB_SHA}" --prod
|
||||
|
||||
- name: Stamp Convex deploy time
|
||||
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
|
||||
run: bunx convex env set APP_DEPLOYED_AT "$(date -u +"%Y-%m-%dT%H:%M:%SZ")" --prod
|
||||
|
||||
- name: Deploy Convex
|
||||
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
|
||||
run: bun run convex:deploy
|
||||
|
||||
- name: Verify Convex contract
|
||||
if: needs.validate-deploy-request.outputs.deploy_backend == 'true'
|
||||
run: bun run verify:convex-contract -- --prod
|
||||
|
||||
wait-vercel-production:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
needs:
|
||||
- preflight-secrets
|
||||
- deploy-convex
|
||||
steps:
|
||||
- name: Wait for Vercel production deployment
|
||||
if: needs.validate-deploy-request.outputs.deploy_frontend == 'true'
|
||||
env:
|
||||
GH_TOKEN: ${{ github.token }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_SHA: ${{ github.sha }}
|
||||
VERCEL_STATUS_CONTEXT: Vercel – clawhub
|
||||
run: |
|
||||
set -euo pipefail
|
||||
for attempt in {1..90}; do
|
||||
state="$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA/status" \
|
||||
if ! state="$(gh api "repos/$GITHUB_REPOSITORY/commits/$GITHUB_SHA/status" \
|
||||
--jq '.statuses[] | select(.context == env.VERCEL_STATUS_CONTEXT) | .state' \
|
||||
2>/dev/null | head -n1)"
|
||||
2>/dev/null | head -n1)"; then
|
||||
echo "GitHub status check failed for $GITHUB_SHA; retrying..."
|
||||
sleep 10
|
||||
continue
|
||||
fi
|
||||
|
||||
case "$state" in
|
||||
success)
|
||||
@@ -104,35 +161,16 @@ jobs:
|
||||
echo "::error::Timed out waiting for Vercel production deployment for $GITHUB_SHA"
|
||||
exit 1
|
||||
|
||||
smoke-production:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 20
|
||||
needs:
|
||||
- preflight-secrets
|
||||
- deploy-convex
|
||||
- wait-vercel-production
|
||||
env:
|
||||
PLAYWRIGHT_BASE_URL: https://clawhub.ai
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- uses: oven-sh/setup-bun@e3914758a49697077f7bcd190d36582a61667aad
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- name: Install
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
- name: Install Playwright browser
|
||||
if: needs.validate-deploy-request.outputs.run_smoke == 'true'
|
||||
run: bunx playwright install --with-deps chromium
|
||||
|
||||
- name: Write authenticated storage state
|
||||
if: env.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON != ''
|
||||
env:
|
||||
PLAYWRIGHT_AUTH_STORAGE_STATE_JSON: ${{ secrets.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON }}
|
||||
if: needs.validate-deploy-request.outputs.run_smoke == 'true' && env.PLAYWRIGHT_AUTH_STORAGE_STATE_JSON != ''
|
||||
run: |
|
||||
echo "$PLAYWRIGHT_AUTH_STORAGE_STATE_JSON" > "$RUNNER_TEMP/playwright-auth.json"
|
||||
echo "PLAYWRIGHT_AUTH_STORAGE_STATE=$RUNNER_TEMP/playwright-auth.json" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Smoke test production
|
||||
if: needs.validate-deploy-request.outputs.run_smoke == 'true'
|
||||
run: bunx playwright test e2e/menu-smoke.pw.test.ts e2e/upload-auth-smoke.pw.test.ts
|
||||
|
||||
@@ -0,0 +1,341 @@
|
||||
name: Package Publish
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
inputs:
|
||||
source:
|
||||
description: Package source to publish. Usually owner/repo, owner/repo@ref, or a GitHub URL.
|
||||
required: false
|
||||
type: string
|
||||
default: ""
|
||||
ref:
|
||||
description: Optional ref to append to the source when source is not already pinned.
|
||||
required: false
|
||||
type: string
|
||||
dry_run:
|
||||
description: Preview only. When true, no publish mutation is performed.
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
json:
|
||||
description: Emit structured JSON output.
|
||||
required: false
|
||||
type: boolean
|
||||
default: true
|
||||
registry:
|
||||
description: ClawHub registry URL.
|
||||
required: false
|
||||
type: string
|
||||
default: https://clawhub.ai
|
||||
site:
|
||||
description: ClawHub site URL.
|
||||
required: false
|
||||
type: string
|
||||
default: https://clawhub.ai
|
||||
owner:
|
||||
description: Optional owner handle override for org/shared publishing.
|
||||
required: false
|
||||
type: string
|
||||
version:
|
||||
description: Optional package version override.
|
||||
required: false
|
||||
type: string
|
||||
tags:
|
||||
description: Optional comma-separated tags override.
|
||||
required: false
|
||||
type: string
|
||||
default: latest
|
||||
source_repo:
|
||||
description: Optional source repo override for local-folder publishes.
|
||||
required: false
|
||||
type: string
|
||||
source_commit:
|
||||
description: Optional source commit override for local-folder publishes.
|
||||
required: false
|
||||
type: string
|
||||
source_ref:
|
||||
description: Optional source ref override for local-folder publishes.
|
||||
required: false
|
||||
type: string
|
||||
clawhub_version:
|
||||
description: Legacy npm CLI version input. Kept for compatibility; the workflow now runs the checked-out source.
|
||||
required: false
|
||||
type: string
|
||||
default: latest
|
||||
secrets:
|
||||
clawhub_token:
|
||||
required: false
|
||||
outputs:
|
||||
publish_json:
|
||||
description: Structured JSON output from clawhub package publish.
|
||||
value: ${{ jobs.publish.outputs.publish_json }}
|
||||
release_id:
|
||||
description: Published release id when dry_run is false.
|
||||
value: ${{ jobs.publish.outputs.release_id }}
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 15
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
outputs:
|
||||
publish_json: ${{ steps.capture.outputs.publish_json }}
|
||||
release_id: ${{ steps.capture.outputs.release_id }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
ref: ${{ github.sha }}
|
||||
|
||||
- uses: oven-sh/setup-bun@e3914758a49697077f7bcd190d36582a61667aad
|
||||
with:
|
||||
bun-version: 1.3.10
|
||||
|
||||
- name: Resolve ClawHub workflow source
|
||||
id: clawhub_source
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from urllib.request import Request, urlopen
|
||||
|
||||
request_token = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_TOKEN", "").strip()
|
||||
request_url = os.environ.get("ACTIONS_ID_TOKEN_REQUEST_URL", "").strip()
|
||||
if not request_token or not request_url:
|
||||
raise SystemExit("GitHub OIDC token request env vars are missing; id-token: write is required.")
|
||||
|
||||
audience = "clawhub-workflow-source"
|
||||
joiner = "&" if "?" in request_url else "?"
|
||||
token_url = f"{request_url}{joiner}audience={audience}"
|
||||
request = Request(
|
||||
token_url,
|
||||
headers={"Authorization": f"Bearer {request_token}"},
|
||||
)
|
||||
with urlopen(request) as response:
|
||||
payload = json.load(response)
|
||||
|
||||
token = str(payload.get("value", "")).strip()
|
||||
if not token:
|
||||
raise SystemExit("GitHub OIDC token response did not include a token value.")
|
||||
|
||||
try:
|
||||
encoded_payload = token.split(".")[1]
|
||||
except IndexError as exc:
|
||||
raise SystemExit("GitHub OIDC token was not a valid JWT.") from exc
|
||||
padding = "=" * (-len(encoded_payload) % 4)
|
||||
claims = json.loads(
|
||||
base64.urlsafe_b64decode(encoded_payload + padding).decode("utf-8")
|
||||
)
|
||||
|
||||
workflow_ref = str(claims.get("job_workflow_ref", "")).strip()
|
||||
workflow_sha = str(claims.get("job_workflow_sha", "")).strip()
|
||||
repo, marker, _ = workflow_ref.partition("/.github/workflows/")
|
||||
if not marker or not repo or not workflow_sha:
|
||||
raise SystemExit(
|
||||
"Unable to resolve reusable workflow source from GitHub OIDC claims: "
|
||||
f"job_workflow_ref={workflow_ref!r} job_workflow_sha={workflow_sha!r}"
|
||||
)
|
||||
|
||||
output_path = Path(os.environ["GITHUB_OUTPUT"])
|
||||
with output_path.open("a", encoding="utf-8") as fh:
|
||||
fh.write(f"repository={repo}\n")
|
||||
fh.write(f"ref={workflow_sha}\n")
|
||||
PY
|
||||
|
||||
- uses: actions/checkout@v6
|
||||
with:
|
||||
repository: ${{ steps.clawhub_source.outputs.repository }}
|
||||
ref: ${{ steps.clawhub_source.outputs.ref }}
|
||||
path: clawhub-source
|
||||
|
||||
- name: Install ClawHub CLI dependencies
|
||||
working-directory: clawhub-source
|
||||
run: bun install --frozen-lockfile
|
||||
|
||||
- name: Validate publish mode inputs
|
||||
env:
|
||||
DRY_RUN: ${{ inputs.dry_run }}
|
||||
JSON_MODE: ${{ inputs.json }}
|
||||
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
run: |
|
||||
if [[ "$JSON_MODE" != "true" ]]; then
|
||||
echo "::warning::This reusable workflow always emits JSON output; forcing --json for downstream parsing."
|
||||
fi
|
||||
if [[ "$DRY_RUN" == "true" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
if [[ -n "$CLAWHUB_TOKEN" ]]; then
|
||||
exit 0
|
||||
fi
|
||||
if [[ "$GITHUB_EVENT_NAME" == "workflow_dispatch" && -n "${ACTIONS_ID_TOKEN_REQUEST_URL:-}" && -n "${ACTIONS_ID_TOKEN_REQUEST_TOKEN:-}" ]]; then
|
||||
echo "No ClawHub token provided; publish will rely on GitHub OIDC trusted publishing."
|
||||
exit 0
|
||||
fi
|
||||
echo "::error::Real publishes need secrets.clawhub_token, or GitHub OIDC on workflow_dispatch runs (permissions.id-token=write)."
|
||||
exit 1
|
||||
|
||||
- name: Write ClawHub config
|
||||
env:
|
||||
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
|
||||
CLAWHUB_REGISTRY: ${{ inputs.registry }}
|
||||
run: |
|
||||
if [[ -z "$CLAWHUB_TOKEN" ]]; then
|
||||
echo "No ClawHub token provided, skipping config file creation."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-config.json"
|
||||
path.write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"registry": os.environ["CLAWHUB_REGISTRY"],
|
||||
"token": os.environ["CLAWHUB_TOKEN"],
|
||||
},
|
||||
indent=2,
|
||||
)
|
||||
+ "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
print(path)
|
||||
PY
|
||||
echo "CLAWHUB_CONFIG_PATH=$RUNNER_TEMP/clawhub-config.json" >> "$GITHUB_ENV"
|
||||
|
||||
- name: Resolve publish command
|
||||
env:
|
||||
INPUT_SOURCE: ${{ inputs.source }}
|
||||
INPUT_REF: ${{ inputs.ref }}
|
||||
INPUT_DRY_RUN: ${{ inputs.dry_run }}
|
||||
INPUT_OWNER: ${{ inputs.owner }}
|
||||
INPUT_VERSION: ${{ inputs.version }}
|
||||
INPUT_TAGS: ${{ inputs.tags }}
|
||||
INPUT_SOURCE_REPO: ${{ inputs.source_repo }}
|
||||
INPUT_SOURCE_COMMIT: ${{ inputs.source_commit }}
|
||||
INPUT_SOURCE_REF: ${{ inputs.source_ref }}
|
||||
INPUT_SITE: ${{ inputs.site }}
|
||||
INPUT_REGISTRY: ${{ inputs.registry }}
|
||||
CLAWHUB_TOKEN: ${{ secrets.clawhub_token }}
|
||||
GITHUB_EVENT_NAME: ${{ github.event_name }}
|
||||
GITHUB_REPOSITORY: ${{ github.repository }}
|
||||
GITHUB_REF: ${{ github.ref }}
|
||||
GITHUB_SHA: ${{ github.sha }}
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
|
||||
source = os.environ["INPUT_SOURCE"].strip()
|
||||
if not source:
|
||||
source = os.environ["GITHUB_REPOSITORY"]
|
||||
source_is_current_repo = source == os.environ["GITHUB_REPOSITORY"]
|
||||
ref = os.environ["INPUT_REF"].strip()
|
||||
if not ref and source_is_current_repo:
|
||||
ref = os.environ["GITHUB_SHA"].strip()
|
||||
is_local_source = source.startswith(".") or source.startswith("/") or Path(source).exists()
|
||||
if ref and "@" not in source and not source.startswith("http") and not is_local_source:
|
||||
source = f"{source}@{ref}"
|
||||
|
||||
cli_entry = (
|
||||
Path(os.environ["GITHUB_WORKSPACE"])
|
||||
/ "clawhub-source"
|
||||
/ "packages"
|
||||
/ "clawhub"
|
||||
/ "src"
|
||||
/ "cli.ts"
|
||||
)
|
||||
if not cli_entry.exists():
|
||||
raise SystemExit(f"Missing ClawHub CLI entrypoint at {cli_entry}")
|
||||
|
||||
cmd = [
|
||||
"bun",
|
||||
str(cli_entry),
|
||||
"package",
|
||||
"publish",
|
||||
source,
|
||||
"--site",
|
||||
os.environ["INPUT_SITE"],
|
||||
"--registry",
|
||||
os.environ["INPUT_REGISTRY"],
|
||||
]
|
||||
|
||||
if os.environ["INPUT_DRY_RUN"] == "true":
|
||||
cmd.append("--dry-run")
|
||||
cmd.append("--json")
|
||||
|
||||
owner = os.environ["INPUT_OWNER"].strip()
|
||||
version = os.environ["INPUT_VERSION"].strip()
|
||||
tags = os.environ["INPUT_TAGS"].strip()
|
||||
if owner:
|
||||
cmd += ["--owner", owner]
|
||||
if version:
|
||||
cmd += ["--version", version]
|
||||
if tags:
|
||||
cmd += ["--tags", tags]
|
||||
source_repo = os.environ["INPUT_SOURCE_REPO"].strip()
|
||||
source_commit = os.environ["INPUT_SOURCE_COMMIT"].strip()
|
||||
source_ref = os.environ["INPUT_SOURCE_REF"].strip()
|
||||
if source_repo:
|
||||
cmd += ["--source-repo", source_repo]
|
||||
if source_commit:
|
||||
cmd += ["--source-commit", source_commit]
|
||||
if source_ref:
|
||||
cmd += ["--source-ref", source_ref]
|
||||
elif source_is_current_repo:
|
||||
github_ref = os.environ["GITHUB_REF"].strip()
|
||||
if github_ref:
|
||||
cmd += ["--source-ref", github_ref]
|
||||
if os.environ["INPUT_DRY_RUN"] != "true" and os.environ["CLAWHUB_TOKEN"].strip():
|
||||
cmd += [
|
||||
"--manual-override-reason",
|
||||
f"GitHub Actions {os.environ['GITHUB_EVENT_NAME'].strip()} publish via CLAWHUB_TOKEN",
|
||||
]
|
||||
|
||||
path = Path(os.environ["RUNNER_TEMP"]) / "clawhub-package-publish-command.sh"
|
||||
shell_line = " ".join(shlex.quote(part) for part in cmd)
|
||||
path.write_text("#!/usr/bin/env bash\nset -euo pipefail\n" + shell_line + "\n", encoding="utf-8")
|
||||
path.chmod(0o755)
|
||||
print(shell_line)
|
||||
PY
|
||||
|
||||
- name: Run package publish
|
||||
run: |
|
||||
set -euo pipefail
|
||||
"$RUNNER_TEMP/clawhub-package-publish-command.sh" | tee "$RUNNER_TEMP/package-publish.json"
|
||||
|
||||
- name: Capture workflow outputs
|
||||
id: capture
|
||||
run: |
|
||||
python3 - <<'PY'
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
output_path = Path(os.environ["RUNNER_TEMP"]) / "package-publish.json"
|
||||
raw = output_path.read_text(encoding="utf-8").strip()
|
||||
parsed = json.loads(raw)
|
||||
|
||||
github_output = Path(os.environ["GITHUB_OUTPUT"])
|
||||
with github_output.open("a", encoding="utf-8") as fh:
|
||||
fh.write("publish_json<<__CLAWHUB_JSON__\n")
|
||||
fh.write(json.dumps(parsed, indent=2))
|
||||
fh.write("\n__CLAWHUB_JSON__\n")
|
||||
release_id = str(parsed.get("releaseId", "") or "")
|
||||
fh.write(f"release_id={release_id}\n")
|
||||
PY
|
||||
|
||||
- name: Upload publish JSON artifact
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: clawhub-package-publish-json
|
||||
path: ${{ runner.temp }}/package-publish.json
|
||||
if-no-files-found: error
|
||||
@@ -1,6 +1,8 @@
|
||||
name: "Security Gate: Secret Scanning"
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ["**"]
|
||||
pull_request:
|
||||
branches: [main, master]
|
||||
|
||||
@@ -16,6 +18,33 @@ jobs:
|
||||
with:
|
||||
fetch-depth: 0 # necessary to support the scoping requirements below
|
||||
|
||||
- name: Resolve scan range
|
||||
id: scan_range
|
||||
env:
|
||||
EVENT_NAME: ${{ github.event_name }}
|
||||
PR_BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
PR_HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
PUSH_BASE_SHA: ${{ github.event.before }}
|
||||
PUSH_HEAD_SHA: ${{ github.sha }}
|
||||
DEFAULT_BRANCH: ${{ github.event.repository.default_branch }}
|
||||
run: |
|
||||
set -euo pipefail
|
||||
zero_sha="0000000000000000000000000000000000000000"
|
||||
|
||||
if [[ "$EVENT_NAME" == "pull_request" ]]; then
|
||||
base="$PR_BASE_SHA"
|
||||
head="$PR_HEAD_SHA"
|
||||
else
|
||||
base="$PUSH_BASE_SHA"
|
||||
head="$PUSH_HEAD_SHA"
|
||||
if [[ -z "$base" || "$base" == "$zero_sha" ]]; then
|
||||
base="origin/$DEFAULT_BRANCH"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "base=$base" >> "$GITHUB_OUTPUT"
|
||||
echo "head=$head" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: TruffleHog OSS
|
||||
id: trufflehog
|
||||
# Use a concrete released ref that resolves in upstream action registry.
|
||||
@@ -23,8 +52,8 @@ jobs:
|
||||
uses: trufflesecurity/trufflehog@v3.93.8
|
||||
with:
|
||||
path: ./
|
||||
base: ${{ github.event.pull_request.base.sha }} # scope it to the committed files
|
||||
head: ${{ github.event.pull_request.head.sha }}
|
||||
base: ${{ steps.scan_range.outputs.base }}
|
||||
head: ${{ steps.scan_range.outputs.head }}
|
||||
extra_args: --only-verified --debug
|
||||
|
||||
- name: Notify on Failure
|
||||
|
||||
@@ -24,3 +24,7 @@ coverage
|
||||
playwright-report
|
||||
test-results
|
||||
.playwright
|
||||
convex/_generated/
|
||||
skills-lock.json
|
||||
*/skills/*
|
||||
skills/*
|
||||
@@ -38,10 +38,22 @@
|
||||
- Commit messages: Conventional Commits (`feat:`, `fix:`, `chore:`, `docs:`…).
|
||||
- Keep changes scoped; avoid repo-wide search/replace.
|
||||
- PRs: include summary + test commands run. Add screenshots for UI changes.
|
||||
- Before merging any PR, verify TypeScript cleanly with `bunx tsc -p packages/schema/tsconfig.json --noEmit` and `bunx tsc -p packages/clawdhub/tsconfig.json --noEmit`; if Convex code changed, also run the repo typecheck path used by deploy so `bunx convex deploy` will not fail on `tsc`.
|
||||
- Before merging any PR, verify TypeScript cleanly with `bunx tsc -p packages/schema/tsconfig.json --noEmit` and `bunx tsc -p packages/clawhub/tsconfig.json --noEmit`; if Convex code changed, also run the repo typecheck path used by deploy so `bunx convex deploy` will not fail on `tsc`.
|
||||
- GitHub comments: for multiline `gh` comments/close messages, use `--body-file`, `--input`, or stdin/heredoc with real newlines; never pass literal `\\n` in shell strings.
|
||||
- Reject PRs that add skills into source code/repo content directly (for example under `skills/` or seed-only additions intended as published skills). Skills must be uploaded/published via CLI.
|
||||
|
||||
## Production Release
|
||||
|
||||
- Production deploys are manual-only. Merging to `main` does **not** deploy.
|
||||
- To release production, start the GitHub Actions `Deploy` workflow from `main`:
|
||||
`gh workflow run deploy.yml --repo openclaw/clawhub --ref main`
|
||||
- The workflow supports `full`, `backend`, and `frontend` targets.
|
||||
- `frontend` currently means: wait for the Vercel production deploy for the selected `main` SHA, then run production smoke checks. It does not call `vercel deploy` directly yet.
|
||||
- The workflow uses the GitHub `Production` environment for deploy secrets, but it does not require a separate approval step.
|
||||
- Prod deploy secrets live on the `Production` environment, not as ordinary repo secrets. Required: `CONVEX_DEPLOY_KEY`. Optional: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON`.
|
||||
- CLI npm releases are also manual-only and tag-based. Stable tags only: `vX.Y.Z`. Start `ClawHub CLI NPM Release` from `main`, first with `preflight_only=true`, then rerun it with the same tag and the successful `preflight_run_id`.
|
||||
- Real CLI publishes wait at the GitHub `npm-release` environment and use npm trusted publishing. Required npm trusted publisher settings: repository `openclaw/clawhub`, workflow `clawhub-cli-npm-release.yml`, environment `npm-release`.
|
||||
|
||||
## Git Notes
|
||||
|
||||
- If `git branch -d/-D <branch>` is policy-blocked, delete the local ref directly: `git update-ref -d refs/heads/<branch>`.
|
||||
@@ -75,3 +87,31 @@
|
||||
- **32K document limit per query.** Split `.collect()` calls by a partition field (e.g., one day at a time instead of a 7-day range). See `rebuildTrendingLeaderboardAction` in `convex/leaderboards.ts` for an example.
|
||||
- **Common mistakes**: `.filter().collect()` without an index; `ctx.db.get()` on large docs in a loop for list views; while loops that paginate the whole table to find filtered results.
|
||||
- **Before writing or reviewing Convex queries, check deployment health.** Run `bunx convex insights` to check for OCC conflicts, `bytesReadLimit`, and `documentsReadLimit` errors. Run `bunx convex logs --failure` to see individual error messages and stack traces. This helps identify which functions are causing bandwidth issues so you can prioritize fixes.
|
||||
|
||||
<!-- convex-ai-start -->
|
||||
This project uses [Convex](https://convex.dev) as its backend.
|
||||
|
||||
When working on Convex code, **always read `convex/_generated/ai/guidelines.md` first** for important guidelines on how to correctly use Convex APIs and patterns. The file contains rules that override what you may have learned about Convex from training data.
|
||||
|
||||
Convex agent skills for common tasks can be installed by running `npx convex ai-files install`.
|
||||
<!-- convex-ai-end -->
|
||||
|
||||
## Stat Field Migration Rules
|
||||
|
||||
The `skills` table maintains two parallel sets of stat fields as part of an in-progress field migration:
|
||||
|
||||
| Legacy (nested, `@deprecated`) | Top-level (source of truth, indexable) |
|
||||
|---|---|
|
||||
| `stats.downloads` | `statsDownloads` |
|
||||
| `stats.stars` | `statsStars` |
|
||||
| `stats.installsCurrent` | `statsInstallsCurrent` |
|
||||
| `stats.installsAllTime` | `statsInstallsAllTime` |
|
||||
|
||||
**Rules:**
|
||||
|
||||
- **Always use `readCanonicalStat(skill, field)` (`convex/lib/skillStats.ts`) to read** any of the four migrated fields. It prefers the top-level field and falls back to the nested field for pre-migration documents. Never access `skill.stats.downloads` / `.stars` / `.installsCurrent` / `.installsAllTime` directly.
|
||||
- **Always use `applySkillStatDeltas()` to write** stat deltas. It writes both the top-level and nested fields in the same patch to keep them in sync.
|
||||
- **Both sets of fields must be written together** in any patch that touches stat values (see the return shape of `applySkillStatDeltas`).
|
||||
- **Nested-only reads are acceptable only for** `stats.comments` and `stats.versions` — no top-level field exists for these yet.
|
||||
- The four legacy nested fields are marked `@deprecated` in `statsValidator` (schema.ts). Any IDE access to `skill.stats.downloads` etc. will show a strikethrough warning — treat this as a signal to use `readCanonicalStat()` instead.
|
||||
- When adding new stat fields, follow the same dual-write pattern and add a cursor-based backfill mutation (see `backfillSkillStatFieldsInternal` for an example).
|
||||
|
||||
@@ -1,5 +1,64 @@
|
||||
# Changelog
|
||||
|
||||
## Unreleased
|
||||
|
||||
### Changed
|
||||
|
||||
- Search: add CJK tokenization support (Chinese/Japanese/Korean) with Intl.Segmenter plus fallback behavior to improve skill query matching (#1596) (thanks @pq-dong).
|
||||
- Stats: centralize migrated skill stat fallback reads through `readCanonicalStat()` and add schema/agent guardrails to discourage direct legacy nested-field access (#1709) (thanks @momothemage).
|
||||
|
||||
### Fixes
|
||||
|
||||
- Stats maintenance: keep skill stat migration fields synchronized by treating top-level stat fields as canonical during backfill/reconcile fallback reads (#1704) (thanks @momothemage).
|
||||
|
||||
## 0.10.0 - 2026-04-05
|
||||
|
||||
### Added
|
||||
|
||||
- Design system: introduce a shared UI component library (`src/components/ui/`) built on Radix UI primitives — Button, Card, Badge, Tabs, Dialog, Input, Textarea, Label, Select, Avatar, Separator, Tooltip, ScrollArea, Sheet, Skeleton, and Table — following the shadcn/ui pattern with `cn()` + Tailwind utilities.
|
||||
- Design system: `Button` supports `asChild` via Radix Slot for polymorphic rendering (e.g., wrapping `<Link>` without extra DOM).
|
||||
- Layout: add `Container` component with `narrow` / `default` / `wide` size presets and `Breadcrumb` component for hierarchical navigation.
|
||||
- Loading: add skeleton loading states (`SkillCardSkeleton`, `SkillDetailSkeleton`, `DashboardSkeleton`) replacing text-based "Loading..." indicators with animated placeholders.
|
||||
- Errors: add `ErrorBoundary` with `resetKey` prop that auto-resets on route changes, wired into the root layout.
|
||||
- Errors: surface fallback messages from Convex API error payloads in mutation/action error toasts.
|
||||
- UX: add `EmptyState` component with icon, headline, description, and optional CTA action used across dashboard, stars, profile, and publish pages.
|
||||
- UX: add confirmation dialogs for destructive skill ownership actions (transfer, abandon).
|
||||
- Markdown: add `MarkdownPreview` component with `react-markdown`, `remark-gfm`, and `react-syntax-highlighter` for rich rendering of skill/plugin READMEs with syntax-highlighted code blocks, GFM tables, and task lists.
|
||||
- Markdown: render tables with the new `Table` UI primitive for consistent styling across skill docs.
|
||||
- Navigation: replace DropdownMenu-based mobile nav with a slide-out `Sheet` panel.
|
||||
- Validation: add Zod schemas (`src/lib/schemas.ts`) for publish-skill, settings, report, and org forms.
|
||||
- Management: restore capability-tags UI (crypto, requires-wallet, can-make-purchases, etc.) that was silently removed during the initial refactor.
|
||||
- Management: add `.catch()` error handling with toast feedback on `setSoftDeleted` calls; prompt for hide/restore reasons.
|
||||
|
||||
### Changed
|
||||
|
||||
- CSS: migrate from a monolithic 5,161-line `styles.css` to Tailwind utilities on components, pruning CSS to ~1,000 lines (81% reduction). Dark mode now uses Tailwind `dark:` variants via a `@variant dark` directive bridging existing CSS custom properties.
|
||||
- Tailwind: add `@theme` block mapping all CSS design tokens (`--bg`, `--surface`, `--ink`, `--accent`, `--line`, `--radius-*`, etc.) into first-class Tailwind utilities.
|
||||
- Pages: modernize all route pages (home, skills browse, skill detail, dashboard, settings, publish-skill, publish-plugin, import, about, CLI auth, stars, souls, user profile, org profile, management, plugins browse, plugin detail) from CSS class selectors to Tailwind + UI primitives.
|
||||
- Skills browse: widen container to `wide` (1400px) for better use of screen space on desktop; same for plugins browse.
|
||||
- Skills browse: replace text-based filter toggles with pill chips and modernize toolbar layout.
|
||||
- Skill detail: migrate tab controls from CSS-styled buttons to Radix `Tabs` primitive with proper `role="tab"` accessibility.
|
||||
- Skill detail: replace inline CSS class-based install card with `SkillInstallCard` using Card + Button primitives.
|
||||
- Header/Footer: migrate from CSS classes to Tailwind utilities with responsive Sheet-based mobile navigation.
|
||||
- Dashboard: replace CSS table layout with `Table` UI primitive; add metric cards and skeleton loading.
|
||||
- Settings: modernize form inputs with `Input`/`Textarea`/`Label` primitives and structured layout.
|
||||
- Publish: use `Dialog` primitive for modals; inline validation indicators; modernized file list display.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Auth: `EmptyState` "Sign in" button on publish page now triggers GitHub OAuth via `useAuthActions` instead of linking to non-existent `/signin` route.
|
||||
- API: fix plugins page dev-mode `{"error":"Only HTML requests are supported here"}` by routing SSR and localhost API fetches directly to the Convex site URL instead of through TanStack Start's request pipeline.
|
||||
- API: fix CORS error when `credentials: "include"` conflicts with `Access-Control-Allow-Origin: *` by making credentials conditional on same-origin requests.
|
||||
- API: fix SSR `packageApiUrl` to always use `VITE_CONVEX_SITE_URL` directly, avoiding `getRequestUrl()` failures when SSR request context is unavailable.
|
||||
- Management: restore `setSoftDeleted` reason parameter for hide/restore actions.
|
||||
- Tests: rename `settings.test.tsx` to `-settings.test.tsx` to exclude from TanStack Router's file-based route discovery.
|
||||
- Tests: add `@convex-dev/auth/react` mock for `useAuthActions` in upload route tests.
|
||||
- Tests: update skill detail tests for Radix tab roles (`role="tab"` instead of `role="button"`), skeleton loading classes (`animate-pulse`), and capability tag data.
|
||||
- Tests: update skills index tests for refreshed UI copy (placeholder text, empty state wording, loading indicator patterns).
|
||||
- Tests: update SkillDiffCard tests for Tailwind active-tab class (`shadow-sm` replacing `.is-active`).
|
||||
- Tests: update packages publish route tests for Tailwind border classes.
|
||||
- Tests: update packageApi tests for conditional credentials and SSR URL resolution.
|
||||
|
||||
## 0.9.0 - 2026-03-23
|
||||
|
||||
### Added
|
||||
|
||||
@@ -30,7 +30,26 @@
|
||||
- NEVER use `--typecheck=disable` on `npx convex deploy`.
|
||||
- Use `npx convex dev --once` to push functions once (not long-running watcher).
|
||||
|
||||
## Production Release
|
||||
|
||||
- Production deploys are manual-only. Merging to `main` does **not** deploy.
|
||||
- Start the GitHub Actions `Deploy` workflow from `main` with `gh workflow run deploy.yml --repo openclaw/clawhub --ref main`.
|
||||
- The workflow supports `full`, `backend`, and `frontend` targets.
|
||||
- `frontend` currently waits for the Vercel production deploy on the selected `main` SHA and then runs smoke checks. It does not trigger Vercel directly yet.
|
||||
- The workflow uses the `Production` environment for deploy secrets, but it does not wait for a separate approval.
|
||||
- Required prod secret: `CONVEX_DEPLOY_KEY` on the `Production` environment. Optional smoke secret: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON`.
|
||||
- CLI npm releases are manual-only and tag-based through `ClawHub CLI NPM Release`. Stable tags only: `vX.Y.Z`. Run a `preflight_only=true` pass first, then rerun with the same tag plus `preflight_run_id` for the real publish.
|
||||
- Real CLI publishes wait at `npm-release` and rely on npm trusted publishing for `openclaw/clawhub` + `clawhub-cli-npm-release.yml` + `npm-release`.
|
||||
|
||||
## Testing
|
||||
|
||||
- Tests use `._handler` to call mutation handlers directly with mock `db` objects.
|
||||
- Mock `db` objects MUST include `normalizeId: vi.fn()` for trigger wrapper compatibility.
|
||||
|
||||
<!-- convex-ai-start -->
|
||||
This project uses [Convex](https://convex.dev) as its backend.
|
||||
|
||||
When working on Convex code, **always read `convex/_generated/ai/guidelines.md` first** for important guidelines on how to correctly use Convex APIs and patterns. The file contains rules that override what you may have learned about Convex from training data.
|
||||
|
||||
Convex agent skills for common tasks can be installed by running `npx convex ai-files install`.
|
||||
<!-- convex-ai-end -->
|
||||
|
||||
+13
-1
@@ -109,7 +109,7 @@ These features degrade gracefully without their keys:
|
||||
|
||||
## CLI Development
|
||||
|
||||
The CLI source lives in [`packages/clawdhub/`](packages/clawdhub/). Both `clawhub` and `clawdhub` are registered as bin aliases.
|
||||
The CLI source lives in [`packages/clawhub/`](packages/clawhub/). Both `clawhub` and `clawdhub` are registered as bin aliases.
|
||||
|
||||
To test the CLI against your local instance:
|
||||
|
||||
@@ -117,6 +117,17 @@ To test the CLI against your local instance:
|
||||
CLAWHUB_REGISTRY=http://127.0.0.1:3210 CLAWHUB_SITE=http://localhost:3000 clawhub search "padel"
|
||||
```
|
||||
|
||||
Use the package-local verification contract when working on the CLI:
|
||||
|
||||
```bash
|
||||
bun run --cwd packages/clawhub test
|
||||
bun run --cwd packages/clawhub verify:build
|
||||
bun run --cwd packages/clawhub test:artifact
|
||||
bun run --cwd packages/clawhub verify
|
||||
```
|
||||
|
||||
`bun test packages/clawhub/` is not the supported workflow. Source tests and built-artifact smoke tests are intentionally split.
|
||||
|
||||
Manual smoke tests are documented in [`docs/manual-testing.md`](docs/manual-testing.md).
|
||||
|
||||
## Skill & Soul Publishing
|
||||
@@ -137,6 +148,7 @@ clawhub publish <path-to-skill-directory>
|
||||
bun run lint # oxlint
|
||||
bun run test # Vitest (80% coverage threshold)
|
||||
bun run build # Vite + Nitro
|
||||
bun run --cwd packages/clawhub verify
|
||||
```
|
||||
|
||||
These are the same checks that run in CI (`.github/workflows/ci.yml`).
|
||||
|
||||
@@ -0,0 +1,356 @@
|
||||
# ClawHub Design System
|
||||
|
||||
This document outlines the design rules, patterns, and guidelines for the ClawHub platform to ensure consistency, accessibility, and maintainability across all components.
|
||||
|
||||
---
|
||||
|
||||
## Color System
|
||||
|
||||
### Brand Palette (OpenClaw)
|
||||
|
||||
ClawHub uses a strict **3-5 color palette** based on the OpenClaw brand:
|
||||
|
||||
| Token | Light Mode | Dark Mode | Usage |
|
||||
|-------|------------|-----------|-------|
|
||||
| `--accent` | `#dc2626` | `#dc2626` | Primary actions, interactive elements, emphasis |
|
||||
| `--accent-deep` | `#b91c1c` | `#ef4444` | Hover states, secondary emphasis |
|
||||
| `--ink` | `#0a0a0a` | `#fafafa` | Primary text |
|
||||
| `--ink-soft` | `#525252` | `#a1a1a1` | Secondary text, descriptions |
|
||||
| `--surface` | `#ffffff` | `#121212` | Card backgrounds, elevated surfaces |
|
||||
| `--bg` | `#fafafa` | `#0a0a0a` | Page background |
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Never exceed 5 colors** without explicit design approval
|
||||
2. **Never use purple/violet prominently** unless explicitly requested
|
||||
3. **Always override text color** when changing background color to ensure contrast
|
||||
4. **Use semantic tokens** (`--accent`, `--ink`, `--surface`) instead of raw colors
|
||||
|
||||
---
|
||||
|
||||
## Typography
|
||||
|
||||
### Font Stack
|
||||
|
||||
```css
|
||||
--font-sans: 'Geist', system-ui, sans-serif;
|
||||
--font-mono: 'Geist Mono', monospace;
|
||||
--font-display: 'Geist', system-ui, sans-serif;
|
||||
```
|
||||
|
||||
### Scale
|
||||
|
||||
| Token | Size | Usage |
|
||||
|-------|------|-------|
|
||||
| `--fs-xs` | 0.75rem (12px) | Labels, badges, metadata |
|
||||
| `--fs-sm` | 0.875rem (14px) | Body text, descriptions |
|
||||
| `--fs-base` | 1rem (16px) | Default body text |
|
||||
| `--fs-md` | 1.125rem (18px) | Subheadings |
|
||||
| `--fs-lg` | 1.25rem (20px) | Section titles |
|
||||
| `--fs-xl` | 1.5rem (24px) | Page headings |
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Maximum 2 font families** per page
|
||||
2. **Line height 1.4-1.6** for body text (use `leading-relaxed`)
|
||||
3. **Never use decorative fonts** for body text
|
||||
4. **Minimum font size: 14px** for readability
|
||||
5. Use `text-balance` or `text-pretty` for titles
|
||||
|
||||
---
|
||||
|
||||
## Layout
|
||||
|
||||
### Method Priority
|
||||
|
||||
Use this hierarchy for layout decisions:
|
||||
|
||||
1. **Flexbox** - Default for most layouts
|
||||
2. **CSS Grid** - Only for complex 2D layouts (cards, galleries)
|
||||
3. **Never use floats** or absolute positioning unless absolutely necessary
|
||||
|
||||
### Spacing Scale
|
||||
|
||||
```css
|
||||
--space-1: 0.25rem /* 4px */
|
||||
--space-2: 0.5rem /* 8px */
|
||||
--space-3: 0.75rem /* 12px */
|
||||
--space-4: 1rem /* 16px */
|
||||
--space-5: 1.5rem /* 24px */
|
||||
--space-6: 2rem /* 32px */
|
||||
```
|
||||
|
||||
### Grid Patterns
|
||||
|
||||
#### Auto-fit Grid (Recommended for Cards)
|
||||
```css
|
||||
grid-template-columns: repeat(auto-fit, minmax(280px, 1fr));
|
||||
```
|
||||
- Automatically adjusts columns based on container width
|
||||
- Prevents orphan items on partial rows
|
||||
- Maintains consistent card widths
|
||||
|
||||
#### Fixed Grid (When exact columns needed)
|
||||
```css
|
||||
/* 3-column at desktop, 2 at tablet, 1 at mobile */
|
||||
grid-template-columns: repeat(3, minmax(0, 1fr));
|
||||
|
||||
@media (max-width: 860px) {
|
||||
grid-template-columns: repeat(2, minmax(0, 1fr));
|
||||
}
|
||||
|
||||
@media (max-width: 520px) {
|
||||
grid-template-columns: 1fr;
|
||||
}
|
||||
```
|
||||
|
||||
### Container Widths
|
||||
|
||||
| Size | Max Width | Usage |
|
||||
|------|-----------|-------|
|
||||
| Default | `--page-max` (1200px) | Standard pages |
|
||||
| Narrow | `--page-narrow` (720px) | Reading content, forms |
|
||||
| Wide | Full width | Dashboards, data tables |
|
||||
|
||||
---
|
||||
|
||||
## Components
|
||||
|
||||
### Cards
|
||||
|
||||
```css
|
||||
.card {
|
||||
padding: var(--space-4);
|
||||
border: 1px solid var(--line);
|
||||
border-radius: var(--r-md);
|
||||
background: var(--surface);
|
||||
}
|
||||
```
|
||||
|
||||
**Rules:**
|
||||
- Always use `display: flex; flex-direction: column;` for consistent height
|
||||
- Add `flex: 1` to content area for equal-height cards in grids
|
||||
- Include hover state with `border-color` and subtle `box-shadow`
|
||||
|
||||
### Buttons
|
||||
|
||||
| Variant | Usage |
|
||||
|---------|-------|
|
||||
| `primary` | Main actions (Submit, Save, Download) |
|
||||
| `secondary` | Alternative actions |
|
||||
| `ghost` | Tertiary actions, navigation |
|
||||
| `destructive` | Delete, remove, dangerous actions |
|
||||
|
||||
**Rules:**
|
||||
- Always include visible focus state
|
||||
- Minimum touch target: 44x44px on mobile
|
||||
- Include `aria-label` when icon-only
|
||||
|
||||
### Form Controls
|
||||
|
||||
- Labels above inputs (not inline)
|
||||
- Error states use `--status-error-fg`
|
||||
- Focus rings use `--accent` with 0.2 opacity
|
||||
- Minimum input height: 40px
|
||||
|
||||
---
|
||||
|
||||
## Responsive Breakpoints
|
||||
|
||||
```css
|
||||
/* Mobile first - base styles for mobile */
|
||||
|
||||
@media (min-width: 520px) {
|
||||
/* Small tablets, large phones */
|
||||
}
|
||||
|
||||
@media (min-width: 640px) {
|
||||
/* Tablets */
|
||||
}
|
||||
|
||||
@media (min-width: 860px) {
|
||||
/* Small desktops, landscape tablets */
|
||||
}
|
||||
|
||||
@media (min-width: 1024px) {
|
||||
/* Desktops */
|
||||
}
|
||||
|
||||
@media (min-width: 1280px) {
|
||||
/* Large desktops */
|
||||
}
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Mobile-first approach** - Base styles target mobile
|
||||
2. **Progressive enhancement** - Add complexity as viewport increases
|
||||
3. **Test intermediate breakpoints** - Avoid jarring layout jumps
|
||||
4. **Never hide critical content** on mobile
|
||||
|
||||
---
|
||||
|
||||
## Accessibility
|
||||
|
||||
### Color Contrast
|
||||
|
||||
- Normal text: Minimum 4.5:1 ratio
|
||||
- Large text (18px+): Minimum 3:1 ratio
|
||||
- Interactive elements: Minimum 3:1 ratio
|
||||
|
||||
### Focus States
|
||||
|
||||
```css
|
||||
:focus-visible {
|
||||
outline: 2px solid var(--accent);
|
||||
outline-offset: 2px;
|
||||
border-radius: 2px;
|
||||
}
|
||||
```
|
||||
|
||||
### Screen Readers
|
||||
|
||||
- Use `sr-only` class for visually hidden but accessible text
|
||||
- Always include `alt` text for images (empty `alt=""` for decorative)
|
||||
- Use semantic HTML elements (`main`, `nav`, `article`, `section`)
|
||||
- Proper heading hierarchy (h1 > h2 > h3, no skipping)
|
||||
|
||||
### Motion
|
||||
|
||||
```css
|
||||
/* Respect user preference */
|
||||
@media (prefers-reduced-motion: reduce) {
|
||||
* {
|
||||
animation-duration: 0.01ms !important;
|
||||
transition-duration: 0.01ms !important;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Animation
|
||||
|
||||
### Timing
|
||||
|
||||
```css
|
||||
--transition-fast: 150ms;
|
||||
--transition-base: 200ms;
|
||||
--transition-slow: 300ms;
|
||||
```
|
||||
|
||||
### Easing
|
||||
|
||||
- Use `ease` or `ease-out` for most transitions
|
||||
- Use `ease-in-out` for enter/exit animations
|
||||
- Never use `linear` except for continuous animations
|
||||
|
||||
### Rules
|
||||
|
||||
1. **Subtle by default** - Avoid flashy animations
|
||||
2. **Purpose-driven** - Animation should provide feedback
|
||||
3. **Respect preferences** - Support `prefers-reduced-motion`
|
||||
4. **Performance** - Use `transform` and `opacity` only
|
||||
|
||||
---
|
||||
|
||||
## Icons
|
||||
|
||||
### Usage
|
||||
|
||||
- Use Lucide icons consistently
|
||||
- Standard sizes: 14px, 16px, 20px, 24px
|
||||
- Include `aria-hidden="true"` for decorative icons
|
||||
- Never use emojis as icons
|
||||
|
||||
### Placement
|
||||
|
||||
- Left of labels in buttons and navigation
|
||||
- Right of labels for external links or dropdowns
|
||||
- Centered when used alone with `aria-label`
|
||||
|
||||
---
|
||||
|
||||
## Dark Mode
|
||||
|
||||
### Implementation
|
||||
|
||||
```css
|
||||
[data-theme="dark"] {
|
||||
/* Dark mode overrides */
|
||||
}
|
||||
```
|
||||
|
||||
### Rules
|
||||
|
||||
1. Never use pure white (`#ffffff`) on dark backgrounds
|
||||
2. Reduce shadow intensity in dark mode
|
||||
3. Adjust image brightness if needed
|
||||
4. Test contrast ratios in both modes
|
||||
|
||||
---
|
||||
|
||||
## Performance
|
||||
|
||||
### CSS
|
||||
|
||||
1. Use CSS custom properties for theming
|
||||
2. Avoid deeply nested selectors (max 3 levels)
|
||||
3. Use `will-change` sparingly
|
||||
4. Prefer `transform` over `top/left` for animations
|
||||
|
||||
### Images
|
||||
|
||||
1. Always specify `width` and `height` attributes
|
||||
2. Use `loading="lazy"` for below-fold images
|
||||
3. Use appropriate formats (WebP with fallbacks)
|
||||
4. Include placeholder or skeleton states
|
||||
|
||||
---
|
||||
|
||||
## Code Style
|
||||
|
||||
### CSS Class Naming
|
||||
|
||||
```css
|
||||
/* Component */
|
||||
.component-name { }
|
||||
|
||||
/* Component modifier */
|
||||
.component-name.variant { }
|
||||
|
||||
/* Component child */
|
||||
.component-name-child { }
|
||||
|
||||
/* State */
|
||||
.component-name.is-active { }
|
||||
.component-name[data-state="open"] { }
|
||||
```
|
||||
|
||||
### File Organization
|
||||
|
||||
```
|
||||
src/
|
||||
components/
|
||||
ui/ # Primitive components (Button, Input, Card)
|
||||
layout/ # Layout components (Container, Header)
|
||||
styles.css # Global styles and design tokens
|
||||
lib/
|
||||
theme.ts # Theme utilities
|
||||
preferences.ts # User preference management
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
Before shipping any UI changes, verify:
|
||||
|
||||
- [ ] Color contrast meets WCAG AA standards
|
||||
- [ ] Focus states are visible
|
||||
- [ ] Layout works at all breakpoints
|
||||
- [ ] Animations respect `prefers-reduced-motion`
|
||||
- [ ] Text is readable at default browser zoom
|
||||
- [ ] Interactive elements have 44px minimum touch target
|
||||
- [ ] Semantic HTML is used appropriately
|
||||
- [ ] Dark mode has been tested
|
||||
@@ -10,7 +10,7 @@
|
||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg?style=for-the-badge" alt="MIT License"></a>
|
||||
</p>
|
||||
|
||||
ClawHub is the **public skill registry for Clawdbot**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
|
||||
ClawHub is the **public skill registry for OpenClaw**: publish, version, and search text-based agent skills (a `SKILL.md` plus supporting files).
|
||||
It's designed for fast browsing + a CLI-friendly API, with moderation hooks and vector search.
|
||||
It also now exposes a native **OpenClaw package catalog** for code plugins and bundle plugins.
|
||||
|
||||
@@ -61,8 +61,8 @@ Common CLI flows:
|
||||
- Browse unified catalog (skills + plugins): `clawhub package explore`, `clawhub package inspect <name>`
|
||||
- Manage local installs: `clawhub install <slug>`, `clawhub uninstall <slug>`, `clawhub list`, `clawhub update --all`
|
||||
- Inspect without installing: `clawhub inspect <slug>`
|
||||
- Publish/sync: `clawhub publish <path>`, `clawhub sync`
|
||||
- Publish plugins: `clawhub package publish <path> [--owner <handle>] --source-repo <owner/repo> --source-commit <sha>`
|
||||
- Publish/sync skills: `clawhub skill publish <path>`, `clawhub sync`
|
||||
- Publish plugins: `clawhub package publish <source>`
|
||||
- Canonicalize owned skills: `clawhub skill rename <slug> <new-slug>`, `clawhub skill merge <source> <target>`
|
||||
|
||||
Docs: [`docs/quickstart.md`](docs/quickstart.md), [`docs/cli.md`](docs/cli.md).
|
||||
|
||||
@@ -3,9 +3,10 @@ import { existsSync } from 'node:fs'
|
||||
import { stat } from 'node:fs/promises'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const distCliUrl = new URL('./packages/clawdhub/dist/cli.js', import.meta.url)
|
||||
const packageRootPath = fileURLToPath(new URL('./packages/clawhub/', import.meta.url))
|
||||
const distCliUrl = new URL('./packages/clawhub/dist/cli.js', import.meta.url)
|
||||
const distCliPath = fileURLToPath(distCliUrl)
|
||||
const srcRootPath = fileURLToPath(new URL('./packages/clawdhub/src/', import.meta.url))
|
||||
const srcRootPath = fileURLToPath(new URL('./packages/clawhub/src/', import.meta.url))
|
||||
|
||||
const shouldBuild = await (async () => {
|
||||
if (!existsSync(distCliPath)) return true
|
||||
@@ -19,7 +20,8 @@ const shouldBuild = await (async () => {
|
||||
})()
|
||||
|
||||
if (shouldBuild) {
|
||||
const proc = Bun.spawn(['bunx', 'tsc', '-p', 'packages/clawdhub/tsconfig.json'], {
|
||||
const proc = Bun.spawn(['bun', 'run', 'build'], {
|
||||
cwd: packageRootPath,
|
||||
stdin: 'inherit',
|
||||
stdout: 'inherit',
|
||||
stderr: 'inherit',
|
||||
@@ -34,6 +36,7 @@ async function getLatestMtime(root: string) {
|
||||
let latest = 0
|
||||
const glob = new Bun.Glob('**/*.ts')
|
||||
for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
|
||||
if (rel.endsWith('.test.ts')) continue
|
||||
const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
|
||||
try {
|
||||
const entry = await stat(path)
|
||||
|
||||
@@ -3,9 +3,10 @@ import { existsSync } from 'node:fs'
|
||||
import { stat } from 'node:fs/promises'
|
||||
import { fileURLToPath } from 'node:url'
|
||||
|
||||
const distCliUrl = new URL('./packages/clawdhub/dist/cli.js', import.meta.url)
|
||||
const packageRootPath = fileURLToPath(new URL('./packages/clawhub/', import.meta.url))
|
||||
const distCliUrl = new URL('./packages/clawhub/dist/cli.js', import.meta.url)
|
||||
const distCliPath = fileURLToPath(distCliUrl)
|
||||
const srcRootPath = fileURLToPath(new URL('./packages/clawdhub/src/', import.meta.url))
|
||||
const srcRootPath = fileURLToPath(new URL('./packages/clawhub/src/', import.meta.url))
|
||||
|
||||
const shouldBuild = await (async () => {
|
||||
if (!existsSync(distCliPath)) return true
|
||||
@@ -19,7 +20,8 @@ const shouldBuild = await (async () => {
|
||||
})()
|
||||
|
||||
if (shouldBuild) {
|
||||
const proc = Bun.spawn(['bunx', 'tsc', '-p', 'packages/clawdhub/tsconfig.json'], {
|
||||
const proc = Bun.spawn(['bun', 'run', 'build'], {
|
||||
cwd: packageRootPath,
|
||||
stdin: 'inherit',
|
||||
stdout: 'inherit',
|
||||
stderr: 'inherit',
|
||||
@@ -34,6 +36,7 @@ async function getLatestMtime(root: string) {
|
||||
let latest = 0
|
||||
const glob = new Bun.Glob('**/*.ts')
|
||||
for await (const rel of glob.scan({ cwd: root, onlyFiles: true })) {
|
||||
if (rel.endsWith('.test.ts')) continue
|
||||
const path = `${root}${root.endsWith('/') ? '' : '/'}${rel}`
|
||||
try {
|
||||
const entry = await stat(path)
|
||||
|
||||
Vendored
+10
@@ -48,6 +48,7 @@ import type * as lib_contentTypes from "../lib/contentTypes.js";
|
||||
import type * as lib_embeddingVisibility from "../lib/embeddingVisibility.js";
|
||||
import type * as lib_embeddings from "../lib/embeddings.js";
|
||||
import type * as lib_githubAccount from "../lib/githubAccount.js";
|
||||
import type * as lib_githubActionsOidc from "../lib/githubActionsOidc.js";
|
||||
import type * as lib_githubBackup from "../lib/githubBackup.js";
|
||||
import type * as lib_githubIdentity from "../lib/githubIdentity.js";
|
||||
import type * as lib_githubImport from "../lib/githubImport.js";
|
||||
@@ -66,6 +67,7 @@ import type * as lib_moderationReasonCodes from "../lib/moderationReasonCodes.js
|
||||
import type * as lib_openaiResponse from "../lib/openaiResponse.js";
|
||||
import type * as lib_packageRegistry from "../lib/packageRegistry.js";
|
||||
import type * as lib_packageSearchDigest from "../lib/packageSearchDigest.js";
|
||||
import type * as lib_packageSecurity from "../lib/packageSecurity.js";
|
||||
import type * as lib_public from "../lib/public.js";
|
||||
import type * as lib_publishLimits from "../lib/publishLimits.js";
|
||||
import type * as lib_publishers from "../lib/publishers.js";
|
||||
@@ -75,6 +77,7 @@ import type * as lib_reservedSlugs from "../lib/reservedSlugs.js";
|
||||
import type * as lib_searchText from "../lib/searchText.js";
|
||||
import type * as lib_securityPrompt from "../lib/securityPrompt.js";
|
||||
import type * as lib_skillBackfill from "../lib/skillBackfill.js";
|
||||
import type * as lib_skillCapabilityTags from "../lib/skillCapabilityTags.js";
|
||||
import type * as lib_skillPublish from "../lib/skillPublish.js";
|
||||
import type * as lib_skillQuality from "../lib/skillQuality.js";
|
||||
import type * as lib_skillSafety from "../lib/skillSafety.js";
|
||||
@@ -91,11 +94,13 @@ import type * as lib_userSearch from "../lib/userSearch.js";
|
||||
import type * as lib_webhooks from "../lib/webhooks.js";
|
||||
import type * as llmEval from "../llmEval.js";
|
||||
import type * as maintenance from "../maintenance.js";
|
||||
import type * as packagePublishTokens from "../packagePublishTokens.js";
|
||||
import type * as packages from "../packages.js";
|
||||
import type * as publishers from "../publishers.js";
|
||||
import type * as rateLimits from "../rateLimits.js";
|
||||
import type * as search from "../search.js";
|
||||
import type * as seed from "../seed.js";
|
||||
import type * as seedDemo from "../seedDemo.js";
|
||||
import type * as seedSouls from "../seedSouls.js";
|
||||
import type * as skillStatEvents from "../skillStatEvents.js";
|
||||
import type * as skillTransfers from "../skillTransfers.js";
|
||||
@@ -160,6 +165,7 @@ declare const fullApi: ApiFromModules<{
|
||||
"lib/embeddingVisibility": typeof lib_embeddingVisibility;
|
||||
"lib/embeddings": typeof lib_embeddings;
|
||||
"lib/githubAccount": typeof lib_githubAccount;
|
||||
"lib/githubActionsOidc": typeof lib_githubActionsOidc;
|
||||
"lib/githubBackup": typeof lib_githubBackup;
|
||||
"lib/githubIdentity": typeof lib_githubIdentity;
|
||||
"lib/githubImport": typeof lib_githubImport;
|
||||
@@ -178,6 +184,7 @@ declare const fullApi: ApiFromModules<{
|
||||
"lib/openaiResponse": typeof lib_openaiResponse;
|
||||
"lib/packageRegistry": typeof lib_packageRegistry;
|
||||
"lib/packageSearchDigest": typeof lib_packageSearchDigest;
|
||||
"lib/packageSecurity": typeof lib_packageSecurity;
|
||||
"lib/public": typeof lib_public;
|
||||
"lib/publishLimits": typeof lib_publishLimits;
|
||||
"lib/publishers": typeof lib_publishers;
|
||||
@@ -187,6 +194,7 @@ declare const fullApi: ApiFromModules<{
|
||||
"lib/searchText": typeof lib_searchText;
|
||||
"lib/securityPrompt": typeof lib_securityPrompt;
|
||||
"lib/skillBackfill": typeof lib_skillBackfill;
|
||||
"lib/skillCapabilityTags": typeof lib_skillCapabilityTags;
|
||||
"lib/skillPublish": typeof lib_skillPublish;
|
||||
"lib/skillQuality": typeof lib_skillQuality;
|
||||
"lib/skillSafety": typeof lib_skillSafety;
|
||||
@@ -203,11 +211,13 @@ declare const fullApi: ApiFromModules<{
|
||||
"lib/webhooks": typeof lib_webhooks;
|
||||
llmEval: typeof llmEval;
|
||||
maintenance: typeof maintenance;
|
||||
packagePublishTokens: typeof packagePublishTokens;
|
||||
packages: typeof packages;
|
||||
publishers: typeof publishers;
|
||||
rateLimits: typeof rateLimits;
|
||||
search: typeof search;
|
||||
seed: typeof seed;
|
||||
seedDemo: typeof seedDemo;
|
||||
seedSouls: typeof seedSouls;
|
||||
skillStatEvents: typeof skillStatEvents;
|
||||
skillTransfers: typeof skillTransfers;
|
||||
|
||||
+18
-2
@@ -10,7 +10,12 @@ function makeCtx({
|
||||
user,
|
||||
banRecords,
|
||||
}: {
|
||||
user: { deletedAt?: number; deactivatedAt?: number; purgedAt?: number } | null;
|
||||
user: {
|
||||
deletedAt?: number;
|
||||
deactivatedAt?: number;
|
||||
purgedAt?: number;
|
||||
banReason?: string;
|
||||
} | null;
|
||||
banRecords?: Array<Record<string, unknown>>;
|
||||
}) {
|
||||
const query = {
|
||||
@@ -112,7 +117,7 @@ describe("handleDeletedUserSignIn", () => {
|
||||
|
||||
it("blocks users auto-banned for malware", async () => {
|
||||
const { ctx } = makeCtx({
|
||||
user: { deletedAt: 123 },
|
||||
user: { deletedAt: 123, banReason: "malware auto-ban" },
|
||||
banRecords: [{ action: "user.autoban.malware" }],
|
||||
});
|
||||
|
||||
@@ -122,4 +127,15 @@ describe("handleDeletedUserSignIn", () => {
|
||||
|
||||
expect(ctx.db.patch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("includes the moderator ban reason in the sign-in error", async () => {
|
||||
const { ctx } = makeCtx({
|
||||
user: { deletedAt: 123, banReason: "Chargeback fraud" },
|
||||
banRecords: [{ action: "user.ban" }],
|
||||
});
|
||||
|
||||
await expect(
|
||||
handleDeletedUserSignIn(ctx as never, { userId, existingUserId: userId }),
|
||||
).rejects.toThrow(`${BANNED_REAUTH_MESSAGE} Reason: Chargeback fraud`);
|
||||
});
|
||||
});
|
||||
|
||||
+16
-3
@@ -7,16 +7,29 @@ import type { DataModel, Id } from "./_generated/dataModel";
|
||||
import { shouldScheduleGitHubProfileSync } from "./lib/githubProfileSync";
|
||||
|
||||
export const BANNED_REAUTH_MESSAGE =
|
||||
"Your account has been banned for uploading malicious skills. If you believe this is a mistake, please contact security@openclaw.ai and we will work with you to restore access.";
|
||||
"This account has been banned and cannot sign in. If you believe this is a mistake, please contact security@openclaw.ai and we will review it.";
|
||||
export const DELETED_ACCOUNT_REAUTH_MESSAGE =
|
||||
"This account has been permanently deleted and cannot be restored.";
|
||||
|
||||
const REAUTH_BLOCKING_BAN_ACTIONS = new Set(["user.ban", "user.autoban.malware"]);
|
||||
|
||||
function getBannedReauthMessage(reason: string | undefined) {
|
||||
const normalizedReason = reason?.trim();
|
||||
if (!normalizedReason || normalizedReason.toLowerCase() === "malware auto-ban") {
|
||||
return BANNED_REAUTH_MESSAGE;
|
||||
}
|
||||
return `${BANNED_REAUTH_MESSAGE} Reason: ${normalizedReason}`;
|
||||
}
|
||||
|
||||
export async function handleDeletedUserSignIn(
|
||||
ctx: GenericMutationCtx<DataModel>,
|
||||
args: { userId: Id<"users">; existingUserId: Id<"users"> | null },
|
||||
userOverride?: { deletedAt?: number; deactivatedAt?: number; purgedAt?: number } | null,
|
||||
userOverride?: {
|
||||
deletedAt?: number;
|
||||
deactivatedAt?: number;
|
||||
purgedAt?: number;
|
||||
banReason?: string;
|
||||
} | null,
|
||||
) {
|
||||
const user = userOverride !== undefined ? userOverride : await ctx.db.get(args.userId);
|
||||
if (!user?.deletedAt && !user?.deactivatedAt) return;
|
||||
@@ -42,7 +55,7 @@ export async function handleDeletedUserSignIn(
|
||||
);
|
||||
|
||||
if (hasBlockingBan) {
|
||||
throw new ConvexError(BANNED_REAUTH_MESSAGE);
|
||||
throw new ConvexError(getBannedReauthMessage(user.banReason));
|
||||
}
|
||||
|
||||
// Migrate legacy self-deleted accounts (stored in deletedAt) to the new
|
||||
|
||||
@@ -58,6 +58,13 @@ crons.interval("vt-cache-backfill", { minutes: 30 }, internal.vt.backfillActiveS
|
||||
batchSize: 100,
|
||||
});
|
||||
|
||||
crons.interval(
|
||||
"package-scan-backfill",
|
||||
{ minutes: 30 },
|
||||
internal.packages.backfillPackageReleaseScansInternal,
|
||||
{ batchSize: 100 },
|
||||
);
|
||||
|
||||
// Daily re-scan of all active skills at 3am UTC
|
||||
crons.daily("vt-daily-rescan", { hourUTC: 3, minuteUTC: 0 }, internal.vt.rescanActiveSkills, {});
|
||||
|
||||
|
||||
@@ -237,6 +237,74 @@ xuezh snapshot --profile default
|
||||
xuezh review next --limit 10
|
||||
xuezh audio process-voice --file ./utterance.wav
|
||||
\`\`\`
|
||||
`,
|
||||
},
|
||||
{
|
||||
slug: "hanzi-helper",
|
||||
displayName: "汉字助手",
|
||||
summary: "汉字学习与分析工具,支持笔画查询、部首检索和组词生成。",
|
||||
version: "0.1.0",
|
||||
metadata: {
|
||||
clawdbot: {
|
||||
nix: {
|
||||
plugin: "github:example/hanzi-helper",
|
||||
systems: ["aarch64-darwin", "x86_64-linux"],
|
||||
},
|
||||
config: {
|
||||
requiredEnv: ["HANZI_DB_PATH"],
|
||||
stateDirs: [".config/hanzi"],
|
||||
example:
|
||||
'config = { env = { HANZI_DB_PATH = ".config/hanzi/db"; }; stateDirs = [ ".config/hanzi" ]; };',
|
||||
},
|
||||
cliHelp: `汉字助手 - Chinese character learning and analysis
|
||||
|
||||
Usage:
|
||||
hanzi-helper [command]
|
||||
|
||||
Available Commands:
|
||||
lookup 查询汉字信息(笔画、部首、释义)
|
||||
radical 按部首检索汉字
|
||||
stroke 按笔画数筛选汉字
|
||||
words 生成汉字组词
|
||||
practice 练习汉字书写
|
||||
quiz 汉字听写测试
|
||||
|
||||
Flags:
|
||||
-h, --help help for hanzi-helper
|
||||
--json Output JSON
|
||||
`,
|
||||
},
|
||||
},
|
||||
rawSkillMd: `---
|
||||
name: hanzi-helper
|
||||
description: 汉字学习与分析工具,提供笔画查询、部首检索、组词生成和汉字听写练习功能。
|
||||
---
|
||||
|
||||
# 汉字助手
|
||||
|
||||
## 功能介绍
|
||||
|
||||
汉字助手是一个强大的中文汉字学习工具,帮助用户深入了解每个汉字的结构和含义。
|
||||
|
||||
## CLI
|
||||
|
||||
\`\`\`bash
|
||||
hanzi-helper lookup --char 学
|
||||
hanzi-helper radical --name 木
|
||||
hanzi-helper stroke --count 8
|
||||
hanzi-helper words --char 大 --limit 20
|
||||
\`\`\`
|
||||
|
||||
## 使用场景
|
||||
|
||||
- **汉字查询**:输入任意汉字,查看笔画数、部首、繁体形式和基本释义
|
||||
- **部首检索**:按部首浏览相关汉字,了解汉字的分类规律
|
||||
- **组词生成**:输入一个汉字,自动生成常用词语和成语
|
||||
- **听写练习**:随机生成汉字听写测试,巩固学习效果
|
||||
|
||||
## 学习建议
|
||||
|
||||
建议每天学习五个新汉字,结合组词和例句加深记忆。坚持使用听写练习功能可以有效提高汉字识别能力。
|
||||
`,
|
||||
},
|
||||
];
|
||||
@@ -401,6 +469,11 @@ export const seedSkillMutation = internalMutation({
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
});
|
||||
await ctx.db.patch(userId, {
|
||||
publishedSkills: 1,
|
||||
totalStars: 0,
|
||||
totalDownloads: 0,
|
||||
});
|
||||
|
||||
const versionId = await ctx.db.insert("skillVersions", {
|
||||
skillId,
|
||||
|
||||
@@ -448,6 +448,22 @@ const EXTRA_SEED_SKILLS: SeedSkillSpec[] = [
|
||||
["SSH_KEY_DIR"],
|
||||
["generate", "rotate", "deploy", "list", "revoke"],
|
||||
),
|
||||
|
||||
// CJK Language Support (2)
|
||||
makeSkill(
|
||||
"nihongo-check",
|
||||
"日本語チェッカー",
|
||||
"日本語文章の文法チェックと翻訳支援ツール。Japanese grammar checker and translation assistant.",
|
||||
["NIHONGO_API_KEY"],
|
||||
["check", "translate", "kanji", "grammar", "vocabulary"],
|
||||
),
|
||||
makeSkill(
|
||||
"hangukgeo-helper",
|
||||
"한국어 도우미",
|
||||
"한국어 학습 보조 도구입니다. Korean language learning assistant with vocabulary and grammar support.",
|
||||
["HANGUL_API_KEY"],
|
||||
["learn", "quiz", "vocabulary", "grammar", "pronunciation"],
|
||||
),
|
||||
];
|
||||
|
||||
function injectMetadata(rawSkillMd: string, metadata: Record<string, unknown>) {
|
||||
|
||||
+42
-19
@@ -1,8 +1,10 @@
|
||||
/* @vitest-environment node */
|
||||
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { internal } from "./_generated/api";
|
||||
import {
|
||||
repointPackageLatestRelease,
|
||||
scheduleOwnerPublisherDigestSync,
|
||||
syncPackageSearchDigestForPackageId,
|
||||
syncPackageSearchDigestsForOwnerUserId,
|
||||
} from "./functions";
|
||||
@@ -60,10 +62,7 @@ describe("package digest sync", () => {
|
||||
},
|
||||
};
|
||||
|
||||
await syncPackageSearchDigestForPackageId(
|
||||
ctx as never,
|
||||
"packages:demo" as never,
|
||||
);
|
||||
await syncPackageSearchDigestForPackageId(ctx as never, "packages:demo" as never);
|
||||
|
||||
expect(ctx.db.insert).toHaveBeenCalledWith(
|
||||
"packageSearchDigest",
|
||||
@@ -126,10 +125,7 @@ describe("package digest sync", () => {
|
||||
},
|
||||
};
|
||||
|
||||
await syncPackageSearchDigestForPackageId(
|
||||
ctx as never,
|
||||
"packages:demo" as never,
|
||||
);
|
||||
await syncPackageSearchDigestForPackageId(ctx as never, "packages:demo" as never);
|
||||
|
||||
expect(ctx.db.insert).toHaveBeenCalledWith(
|
||||
"packageSearchDigest",
|
||||
@@ -441,13 +437,11 @@ describe("package digest sync", () => {
|
||||
updatedAt: 2,
|
||||
verification: undefined,
|
||||
};
|
||||
const paginate = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
page: [pkg],
|
||||
isDone: true,
|
||||
continueCursor: "",
|
||||
});
|
||||
const paginate = vi.fn().mockResolvedValueOnce({
|
||||
page: [pkg],
|
||||
isDone: true,
|
||||
continueCursor: "",
|
||||
});
|
||||
const ctx = {
|
||||
db: {
|
||||
get: vi.fn(async (id: string) => {
|
||||
@@ -486,10 +480,7 @@ describe("package digest sync", () => {
|
||||
},
|
||||
};
|
||||
|
||||
await syncPackageSearchDigestsForOwnerUserId(
|
||||
ctx as never,
|
||||
"users:owner" as never,
|
||||
);
|
||||
await syncPackageSearchDigestsForOwnerUserId(ctx as never, "users:owner" as never);
|
||||
|
||||
expect(paginate).toHaveBeenCalledWith({ cursor: null, numItems: 100 });
|
||||
expect(ctx.db.insert).toHaveBeenCalledWith(
|
||||
@@ -501,3 +492,35 @@ describe("package digest sync", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("publisher digest scheduling", () => {
|
||||
it("schedules package and skill digest sync in separate background mutations", async () => {
|
||||
const ctx = {
|
||||
scheduler: {
|
||||
runAfter: vi.fn().mockResolvedValue(undefined),
|
||||
},
|
||||
};
|
||||
|
||||
await scheduleOwnerPublisherDigestSync(ctx as never, "publishers:demo" as never);
|
||||
|
||||
expect(ctx.scheduler.runAfter).toHaveBeenCalledTimes(2);
|
||||
expect(ctx.scheduler.runAfter).toHaveBeenNthCalledWith(
|
||||
1,
|
||||
0,
|
||||
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
|
||||
{ ownerPublisherId: "publishers:demo" },
|
||||
);
|
||||
expect(ctx.scheduler.runAfter).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
0,
|
||||
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
|
||||
{ ownerPublisherId: "publishers:demo" },
|
||||
);
|
||||
});
|
||||
|
||||
it("skips scheduling when the trigger context has no scheduler", async () => {
|
||||
await expect(
|
||||
scheduleOwnerPublisherDigestSync({} as never, "publishers:demo" as never),
|
||||
).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
+46
-6
@@ -1,6 +1,8 @@
|
||||
import { customCtx, customMutation } from "convex-helpers/server/customFunctions";
|
||||
import { Triggers } from "convex-helpers/server/triggers";
|
||||
import { v } from "convex/values";
|
||||
import semver from "semver";
|
||||
import { internal } from "./_generated/api";
|
||||
import type { DataModel, Doc, Id } from "./_generated/dataModel";
|
||||
import {
|
||||
mutation as rawMutation,
|
||||
@@ -30,6 +32,7 @@ function isMissingTableError(error: unknown, table: string) {
|
||||
}
|
||||
|
||||
type PackageDigestSyncCtx = Pick<MutationCtx, "db">;
|
||||
type OwnerPublisherDigestScheduleCtx = Pick<Partial<MutationCtx>, "scheduler">;
|
||||
type LatestPackageRelease = Pick<
|
||||
Doc<"packageReleases">,
|
||||
| "_id"
|
||||
@@ -125,7 +128,8 @@ async function syncPackageSearchDigest(
|
||||
});
|
||||
await upsertPackageSearchDigest(ctx, {
|
||||
...fields,
|
||||
latestVersion: latestRelease && !latestRelease.softDeletedAt ? latestRelease.version : undefined,
|
||||
latestVersion:
|
||||
latestRelease && !latestRelease.softDeletedAt ? latestRelease.version : undefined,
|
||||
ownerHandle: owner?.handle ?? "",
|
||||
ownerKind: owner?.kind,
|
||||
});
|
||||
@@ -233,6 +237,41 @@ export async function syncSkillSearchDigestsForOwnerPublisherId(
|
||||
}
|
||||
}
|
||||
|
||||
export async function scheduleOwnerPublisherDigestSync(
|
||||
ctx: OwnerPublisherDigestScheduleCtx,
|
||||
ownerPublisherId: Id<"publishers"> | null | undefined,
|
||||
) {
|
||||
if (!ownerPublisherId || !ctx.scheduler) return;
|
||||
await ctx.scheduler.runAfter(
|
||||
0,
|
||||
internal.functions.syncPackageSearchDigestsForOwnerPublisherIdInternal,
|
||||
{ ownerPublisherId },
|
||||
);
|
||||
await ctx.scheduler.runAfter(
|
||||
0,
|
||||
internal.functions.syncSkillSearchDigestsForOwnerPublisherIdInternal,
|
||||
{ ownerPublisherId },
|
||||
);
|
||||
}
|
||||
|
||||
export const syncPackageSearchDigestsForOwnerPublisherIdInternal = rawInternalMutation({
|
||||
args: {
|
||||
ownerPublisherId: v.id("publishers"),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
await syncPackageSearchDigestsForOwnerPublisherId(ctx, args.ownerPublisherId);
|
||||
},
|
||||
});
|
||||
|
||||
export const syncSkillSearchDigestsForOwnerPublisherIdInternal = rawInternalMutation({
|
||||
args: {
|
||||
ownerPublisherId: v.id("publishers"),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
await syncSkillSearchDigestsForOwnerPublisherId(ctx, args.ownerPublisherId);
|
||||
},
|
||||
});
|
||||
|
||||
export async function repointPackageLatestRelease(
|
||||
ctx: PackageDigestSyncCtx,
|
||||
packageId: Id<"packages"> | null | undefined,
|
||||
@@ -306,13 +345,15 @@ triggers.register("packages", async (ctx, change) => {
|
||||
|
||||
triggers.register("packageReleases", async (ctx, change) => {
|
||||
if (change.operation === "insert") return;
|
||||
if (change.operation === "update" && change.oldDoc.softDeletedAt === change.newDoc.softDeletedAt) {
|
||||
if (
|
||||
change.operation === "update" &&
|
||||
change.oldDoc.softDeletedAt === change.newDoc.softDeletedAt
|
||||
) {
|
||||
return;
|
||||
}
|
||||
const packageId =
|
||||
change.operation === "delete" ? change.oldDoc.packageId : change.newDoc.packageId;
|
||||
const affectedReleaseId =
|
||||
change.operation === "delete" ? change.oldDoc._id : change.newDoc._id;
|
||||
const affectedReleaseId = change.operation === "delete" ? change.oldDoc._id : change.newDoc._id;
|
||||
if (change.operation === "delete" || change.newDoc.softDeletedAt) {
|
||||
await repointPackageLatestRelease(ctx, packageId, affectedReleaseId);
|
||||
return;
|
||||
@@ -335,8 +376,7 @@ triggers.register("users", async (ctx, change) => {
|
||||
|
||||
triggers.register("publishers", async (ctx, change) => {
|
||||
const ownerPublisherId = change.operation === "delete" ? change.id : change.newDoc._id;
|
||||
await syncPackageSearchDigestsForOwnerPublisherId(ctx, ownerPublisherId);
|
||||
await syncSkillSearchDigestsForOwnerPublisherId(ctx, ownerPublisherId);
|
||||
await scheduleOwnerPublisherDigestSync(ctx, ownerPublisherId);
|
||||
});
|
||||
|
||||
export const mutation = customMutation(rawMutation, customCtx(triggers.wrapDB));
|
||||
|
||||
+36
-1
@@ -17,11 +17,16 @@ import {
|
||||
listBundlePluginsV1Http,
|
||||
listCodePluginsV1Http,
|
||||
listPackagesV1Http,
|
||||
listPluginsV1Http,
|
||||
listSkillsV1Http,
|
||||
listSoulsV1Http,
|
||||
mintPublishTokenV1Http,
|
||||
packagesDeleteRouterV1Http,
|
||||
packagesGetRouterV1Http,
|
||||
publishSkillV1Http,
|
||||
packagesPostRouterV1Http,
|
||||
pluginsGetRouterV1Http,
|
||||
publishPackageV1Http,
|
||||
publishSkillV1Http,
|
||||
publishSoulV1Http,
|
||||
resolveSkillVersionV1Http,
|
||||
searchSkillsV1Http,
|
||||
@@ -74,6 +79,12 @@ http.route({
|
||||
handler: listPackagesV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
path: ApiRoutes.plugins,
|
||||
method: "GET",
|
||||
handler: listPluginsV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
path: ApiRoutes.codePlugins,
|
||||
method: "GET",
|
||||
@@ -98,6 +109,12 @@ http.route({
|
||||
handler: packagesGetRouterV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
pathPrefix: `${ApiRoutes.plugins}/`,
|
||||
method: "GET",
|
||||
handler: pluginsGetRouterV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
path: ApiRoutes.skills,
|
||||
method: "POST",
|
||||
@@ -110,6 +127,24 @@ http.route({
|
||||
handler: publishPackageV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
path: ApiRoutes.publishTokenMint,
|
||||
method: "POST",
|
||||
handler: mintPublishTokenV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
pathPrefix: `${ApiRoutes.packages}/`,
|
||||
method: "POST",
|
||||
handler: packagesPostRouterV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
pathPrefix: `${ApiRoutes.packages}/`,
|
||||
method: "DELETE",
|
||||
handler: packagesDeleteRouterV1Http,
|
||||
});
|
||||
|
||||
http.route({
|
||||
pathPrefix: `${ApiRoutes.skills}/`,
|
||||
method: "POST",
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -3,7 +3,12 @@ import {
|
||||
listBundlePluginsV1Handler,
|
||||
listCodePluginsV1Handler,
|
||||
listPackagesV1Handler,
|
||||
listPluginsV1Handler,
|
||||
mintPublishTokenV1Handler,
|
||||
packagesDeleteRouterV1Handler,
|
||||
packagesGetRouterV1Handler,
|
||||
packagesPostRouterV1Handler,
|
||||
pluginsGetRouterV1Handler,
|
||||
publishPackageV1Handler,
|
||||
} from "./httpApiV1/packagesV1";
|
||||
import {
|
||||
@@ -28,8 +33,13 @@ import { usersListV1Handler, usersPostRouterV1Handler } from "./httpApiV1/usersV
|
||||
import { whoamiV1Handler } from "./httpApiV1/whoamiV1";
|
||||
|
||||
export const listPackagesV1Http = httpAction(listPackagesV1Handler);
|
||||
export const listPluginsV1Http = httpAction(listPluginsV1Handler);
|
||||
export const packagesGetRouterV1Http = httpAction(packagesGetRouterV1Handler);
|
||||
export const packagesPostRouterV1Http = httpAction(packagesPostRouterV1Handler);
|
||||
export const packagesDeleteRouterV1Http = httpAction(packagesDeleteRouterV1Handler);
|
||||
export const pluginsGetRouterV1Http = httpAction(pluginsGetRouterV1Handler);
|
||||
export const publishPackageV1Http = httpAction(publishPackageV1Handler);
|
||||
export const mintPublishTokenV1Http = httpAction(mintPublishTokenV1Handler);
|
||||
export const listCodePluginsV1Http = httpAction(listCodePluginsV1Handler);
|
||||
export const listBundlePluginsV1Http = httpAction(listBundlePluginsV1Handler);
|
||||
|
||||
@@ -57,8 +67,13 @@ export const usersListV1Http = httpAction(usersListV1Handler);
|
||||
|
||||
export const __handlers = {
|
||||
listPackagesV1Handler,
|
||||
listPluginsV1Handler,
|
||||
packagesGetRouterV1Handler,
|
||||
packagesPostRouterV1Handler,
|
||||
packagesDeleteRouterV1Handler,
|
||||
pluginsGetRouterV1Handler,
|
||||
publishPackageV1Handler,
|
||||
mintPublishTokenV1Handler,
|
||||
listCodePluginsV1Handler,
|
||||
listBundlePluginsV1Handler,
|
||||
searchSkillsV1Handler,
|
||||
|
||||
+566
-212
File diff suppressed because it is too large
Load Diff
@@ -1,9 +1,9 @@
|
||||
import { CliPublishRequestSchema, parseArk } from "clawhub-schema";
|
||||
import { CliPublishRequestSchema, normalizeTextContentType, parseArk } from "clawhub-schema";
|
||||
import { internal } from "../_generated/api";
|
||||
import type { Doc, Id } from "../_generated/dataModel";
|
||||
import type { ActionCtx } from "../_generated/server";
|
||||
import { assertAdmin } from "../lib/access";
|
||||
import { requireApiTokenUser } from "../lib/apiTokenAuth";
|
||||
import { requireApiTokenUser, requirePackagePublishAuth } from "../lib/apiTokenAuth";
|
||||
import { corsHeaders, mergeHeaders } from "../lib/httpHeaders";
|
||||
import { getPublishFileSizeError, MAX_PUBLISH_FILE_BYTES } from "../lib/publishLimits";
|
||||
import { isMacJunkPath } from "../lib/skills";
|
||||
@@ -25,7 +25,9 @@ export function safeTextFileResponse(params: {
|
||||
size: number;
|
||||
headers?: HeadersInit;
|
||||
}) {
|
||||
const isSvg = isSvgLike(params.contentType, params.path);
|
||||
const contentType =
|
||||
normalizeTextContentType(params.path, params.contentType) ?? params.contentType;
|
||||
const isSvg = isSvgLike(contentType, params.path);
|
||||
|
||||
// For any text response that a browser might try to render, lock it down.
|
||||
// In particular, this prevents SVG <foreignObject> script execution from reading
|
||||
@@ -33,8 +35,8 @@ export function safeTextFileResponse(params: {
|
||||
const headers = mergeHeaders(
|
||||
params.headers,
|
||||
{
|
||||
"Content-Type": params.contentType
|
||||
? `${params.contentType}; charset=utf-8`
|
||||
"Content-Type": contentType
|
||||
? `${contentType}; charset=utf-8`
|
||||
: "text/plain; charset=utf-8",
|
||||
"Cache-Control": "private, max-age=60",
|
||||
ETag: params.sha256,
|
||||
@@ -101,6 +103,18 @@ export async function requireApiTokenUserOrResponse(
|
||||
}
|
||||
}
|
||||
|
||||
export async function requirePackagePublishAuthOrResponse(
|
||||
ctx: ActionCtx,
|
||||
request: Request,
|
||||
headers: HeadersInit,
|
||||
) {
|
||||
try {
|
||||
return { ok: true as const, auth: await requirePackagePublishAuth(ctx, request) };
|
||||
} catch {
|
||||
return { ok: false as const, response: text("Unauthorized", 401, headers) };
|
||||
}
|
||||
}
|
||||
|
||||
export function requireAdminOrResponse(user: Doc<"users">, headers: HeadersInit) {
|
||||
try {
|
||||
assertAdmin(user);
|
||||
|
||||
@@ -1,9 +1,11 @@
|
||||
import { api, internal } from "../_generated/api";
|
||||
import { normalizeTextContentType } from "clawhub-schema";
|
||||
import type { Doc, Id } from "../_generated/dataModel";
|
||||
import type { ActionCtx } from "../_generated/server";
|
||||
import { getOptionalApiTokenUserId, requireApiTokenUser } from "../lib/apiTokenAuth";
|
||||
import { applyRateLimit, parseBearerToken } from "../lib/httpRateLimit";
|
||||
import { parseBooleanQueryParam, resolveBooleanQueryParam } from "../lib/httpUtils";
|
||||
import type { LlmEvalDimension } from "../lib/securityPrompt";
|
||||
import { publishVersionForUser } from "../skills";
|
||||
import {
|
||||
MAX_RAW_FILE_BYTES,
|
||||
@@ -69,6 +71,11 @@ type PublicSkillVersionParsed = {
|
||||
clawdis?: { os?: string[]; nix?: { plugin?: boolean; systems?: string[] } };
|
||||
};
|
||||
|
||||
type PublicSkillVersionStaticScan = Pick<
|
||||
NonNullable<Doc<"skillVersions">["staticScan"]>,
|
||||
"status" | "reasonCodes" | "summary" | "engineVersion" | "checkedAt"
|
||||
>;
|
||||
|
||||
type PublicSkillVersionResponse = {
|
||||
_id: Id<"skillVersions">;
|
||||
version: string;
|
||||
@@ -81,6 +88,8 @@ type PublicSkillVersionResponse = {
|
||||
sha256hash?: string;
|
||||
vtAnalysis?: Doc<"skillVersions">["vtAnalysis"];
|
||||
llmAnalysis?: Doc<"skillVersions">["llmAnalysis"];
|
||||
staticScan?: PublicSkillVersionStaticScan;
|
||||
capabilityTags?: string[];
|
||||
};
|
||||
|
||||
type ModerationEvidence = {
|
||||
@@ -189,7 +198,16 @@ type SkillSecuritySnapshot = {
|
||||
hasScanResult: boolean;
|
||||
sha256hash: string | null;
|
||||
virustotalUrl: string | null;
|
||||
capabilityTags: string[];
|
||||
scanners: {
|
||||
static: {
|
||||
status: string;
|
||||
normalizedStatus: NormalizedSecurityStatus;
|
||||
reasonCodes: string[];
|
||||
summary: string | null;
|
||||
engineVersion: string | null;
|
||||
checkedAt: number | null;
|
||||
} | null;
|
||||
vt: {
|
||||
status: string;
|
||||
verdict: string | null;
|
||||
@@ -204,7 +222,7 @@ type SkillSecuritySnapshot = {
|
||||
normalizedStatus: NormalizedSecurityStatus;
|
||||
confidence: string | null;
|
||||
summary: string | null;
|
||||
dimensions: NonNullable<Doc<"skillVersions">["llmAnalysis"]>["dimensions"] | null;
|
||||
dimensions: LlmEvalDimension[] | null;
|
||||
guidance: string | null;
|
||||
findings: string | null;
|
||||
model: string | null;
|
||||
@@ -260,7 +278,7 @@ function mergeSecurityStatuses(statuses: NormalizedSecurityStatus[]) {
|
||||
}
|
||||
|
||||
function hasLlmDimensionWarnings(
|
||||
dimensions: NonNullable<Doc<"skillVersions">["llmAnalysis"]>["dimensions"] | undefined,
|
||||
dimensions: LlmEvalDimension[] | undefined,
|
||||
) {
|
||||
if (!Array.isArray(dimensions)) return false;
|
||||
return dimensions.some((dimension) => {
|
||||
@@ -271,28 +289,37 @@ function hasLlmDimensionWarnings(
|
||||
}
|
||||
|
||||
function buildSkillSecuritySnapshot(
|
||||
version: Pick<PublicSkillVersionResponse, "sha256hash" | "vtAnalysis" | "llmAnalysis">,
|
||||
version: Pick<
|
||||
PublicSkillVersionResponse,
|
||||
"sha256hash" | "vtAnalysis" | "llmAnalysis" | "staticScan" | "capabilityTags"
|
||||
>,
|
||||
): SkillSecuritySnapshot | null {
|
||||
const capabilityTags = version.capabilityTags ?? [];
|
||||
const sha256hash = version.sha256hash ?? null;
|
||||
const vt = version.vtAnalysis;
|
||||
const llm = version.llmAnalysis;
|
||||
const staticScan = version.staticScan;
|
||||
|
||||
if (!sha256hash && !vt && !llm) return null;
|
||||
if (!sha256hash && !vt && !llm && !staticScan && capabilityTags.length === 0) return null;
|
||||
|
||||
const staticStatus = staticScan ? normalizeSecurityStatus(staticScan.status) : null;
|
||||
const vtStatus = vt ? normalizeSecurityStatus(vt.verdict ?? vt.status) : null;
|
||||
const llmStatus = llm ? normalizeSecurityStatus(llm.verdict ?? llm.status) : null;
|
||||
|
||||
const statuses: NormalizedSecurityStatus[] = [];
|
||||
if (staticStatus) statuses.push(staticStatus);
|
||||
if (vtStatus) statuses.push(vtStatus);
|
||||
if (llmStatus) statuses.push(llmStatus);
|
||||
if (statuses.length === 0 && sha256hash) statuses.push("pending");
|
||||
const status = mergeSecurityStatuses(statuses);
|
||||
const hasScanResult =
|
||||
isDefinitiveSecurityStatus(vtStatus) || isDefinitiveSecurityStatus(llmStatus);
|
||||
isDefinitiveSecurityStatus(staticStatus) ||
|
||||
isDefinitiveSecurityStatus(vtStatus) ||
|
||||
isDefinitiveSecurityStatus(llmStatus);
|
||||
const hasWarnings =
|
||||
status === "suspicious" || status === "malicious" || hasLlmDimensionWarnings(llm?.dimensions);
|
||||
|
||||
const checkedAtCandidates = [vt?.checkedAt, llm?.checkedAt].filter(
|
||||
const checkedAtCandidates = [staticScan?.checkedAt, vt?.checkedAt, llm?.checkedAt].filter(
|
||||
(value): value is number => typeof value === "number",
|
||||
);
|
||||
const checkedAt = checkedAtCandidates.length > 0 ? Math.max(...checkedAtCandidates) : null;
|
||||
@@ -305,7 +332,18 @@ function buildSkillSecuritySnapshot(
|
||||
hasScanResult,
|
||||
sha256hash,
|
||||
virustotalUrl: sha256hash ? `https://www.virustotal.com/gui/file/${sha256hash}` : null,
|
||||
capabilityTags,
|
||||
scanners: {
|
||||
static: staticScan
|
||||
? {
|
||||
status: staticScan.status,
|
||||
normalizedStatus: staticStatus ?? "pending",
|
||||
reasonCodes: staticScan.reasonCodes ?? [],
|
||||
summary: staticScan.summary ?? null,
|
||||
engineVersion: staticScan.engineVersion ?? null,
|
||||
checkedAt: staticScan.checkedAt ?? null,
|
||||
}
|
||||
: null,
|
||||
vt: vt
|
||||
? {
|
||||
status: vt.status,
|
||||
@@ -726,7 +764,7 @@ export async function skillsGetRouterV1Handler(ctx: ActionCtx, request: Request)
|
||||
path: file.path,
|
||||
size: file.size,
|
||||
sha256: file.sha256,
|
||||
contentType: file.contentType ?? null,
|
||||
contentType: normalizeTextContentType(file.path, file.contentType) ?? null,
|
||||
})),
|
||||
security: security ?? undefined,
|
||||
},
|
||||
|
||||
@@ -22,7 +22,10 @@ export async function starsPostRouterV1Handler(ctx: ActionCtx, request: Request)
|
||||
skillId: skill._id,
|
||||
});
|
||||
return json(result, 200, rate.headers);
|
||||
} catch {
|
||||
} catch (e) {
|
||||
if (e instanceof Error && e.message === "Skill not found") {
|
||||
return text("Skill not found", 404, rate.headers);
|
||||
}
|
||||
return text("Unauthorized", 401, rate.headers);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -236,9 +236,9 @@ async function handleAdminEnsurePublisher(
|
||||
const handle = typeof payload.handle === "string" ? payload.handle.trim().toLowerCase() : "";
|
||||
if (!handle) return text("Missing handle", 400, headers);
|
||||
|
||||
const displayName = typeof payload.displayName === "string" ? payload.displayName.trim() : undefined;
|
||||
const trusted =
|
||||
typeof payload.trusted === "boolean" ? payload.trusted : true;
|
||||
const displayName =
|
||||
typeof payload.displayName === "string" ? payload.displayName.trim() : undefined;
|
||||
const trusted = typeof payload.trusted === "boolean" ? payload.trusted : true;
|
||||
|
||||
try {
|
||||
const result = await ctx.runMutation(internal.publishers.ensureOrgPublisherHandleInternal, {
|
||||
|
||||
@@ -36,6 +36,17 @@ describe("access.requireUser", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("throws when auth resolves to an invalid user id", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:broken" as never);
|
||||
const dbGet = vi.fn().mockRejectedValue(new Error("Table mismatch"));
|
||||
|
||||
await expect(
|
||||
requireUser({
|
||||
db: { get: dbGet },
|
||||
} as never),
|
||||
).rejects.toThrow("User not found");
|
||||
});
|
||||
|
||||
it("returns auth user when active", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:2" as never);
|
||||
const user = { _id: "users:2", role: "user" };
|
||||
@@ -77,6 +88,17 @@ describe("access.requireUserFromAction", () => {
|
||||
}
|
||||
});
|
||||
|
||||
it("throws when action auth resolves to an invalid user id", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:broken" as never);
|
||||
const runQuery = vi.fn().mockRejectedValue(new Error("Table mismatch"));
|
||||
|
||||
await expect(
|
||||
requireUserFromAction({
|
||||
runQuery,
|
||||
} as never),
|
||||
).rejects.toThrow("User not found");
|
||||
});
|
||||
|
||||
it("returns active user from action query", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:9" as never);
|
||||
const user = { _id: "users:9", role: "admin" };
|
||||
|
||||
+40
-2
@@ -5,10 +5,43 @@ import type { ActionCtx, MutationCtx, QueryCtx } from "../_generated/server";
|
||||
|
||||
export type Role = "admin" | "moderator" | "user";
|
||||
|
||||
export async function getOptionalActiveAuthUserId(
|
||||
ctx: MutationCtx | QueryCtx,
|
||||
): Promise<Id<"users"> | undefined> {
|
||||
try {
|
||||
const userId = await getAuthUserId(ctx);
|
||||
if (!userId) return undefined;
|
||||
const user = await ctx.db.get(userId);
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return undefined;
|
||||
return userId;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export async function getOptionalActiveAuthUserIdFromAction(
|
||||
ctx: ActionCtx,
|
||||
): Promise<Id<"users"> | undefined> {
|
||||
try {
|
||||
const userId = await getAuthUserId(ctx);
|
||||
if (!userId) return undefined;
|
||||
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId });
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return undefined;
|
||||
return userId;
|
||||
} catch {
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export async function requireUser(ctx: MutationCtx | QueryCtx) {
|
||||
const userId = await getAuthUserId(ctx);
|
||||
if (!userId) throw new Error("Unauthorized");
|
||||
const user = await ctx.db.get(userId);
|
||||
let user: Doc<"users"> | null;
|
||||
try {
|
||||
user = await ctx.db.get(userId);
|
||||
} catch {
|
||||
throw new Error("User not found");
|
||||
}
|
||||
if (!user || user.deletedAt || user.deactivatedAt) throw new Error("User not found");
|
||||
return { userId, user };
|
||||
}
|
||||
@@ -18,7 +51,12 @@ export async function requireUserFromAction(
|
||||
): Promise<{ userId: Id<"users">; user: Doc<"users"> }> {
|
||||
const userId = await getAuthUserId(ctx);
|
||||
if (!userId) throw new Error("Unauthorized");
|
||||
const user = await ctx.runQuery(internal.users.getByIdInternal, { userId });
|
||||
let user: Doc<"users"> | null;
|
||||
try {
|
||||
user = await ctx.runQuery(internal.users.getByIdInternal, { userId });
|
||||
} catch {
|
||||
throw new Error("User not found");
|
||||
}
|
||||
if (!user || user.deletedAt || user.deactivatedAt) throw new Error("User not found");
|
||||
return { userId, user: user as Doc<"users"> };
|
||||
}
|
||||
|
||||
+81
-10
@@ -5,6 +5,29 @@ import type { ActionCtx } from "../_generated/server";
|
||||
import { hashToken } from "./tokens";
|
||||
|
||||
type TokenAuthResult = { user: Doc<"users">; userId: Doc<"users">["_id"] };
|
||||
type ApiTokenDoc = Doc<"apiTokens">;
|
||||
type PackagePublishTokenAuthResult = {
|
||||
kind: "github-actions";
|
||||
publishToken: Doc<"packagePublishTokens">;
|
||||
};
|
||||
type PackagePublishTokenDoc = Doc<"packagePublishTokens">;
|
||||
type UserPackagePublishAuthResult = {
|
||||
kind: "user";
|
||||
user: Doc<"users">;
|
||||
userId: Doc<"users">["_id"];
|
||||
};
|
||||
|
||||
const internalRefs = internal as unknown as {
|
||||
tokens: {
|
||||
getByHashInternal: unknown;
|
||||
getUserForTokenInternal: unknown;
|
||||
touchInternal: unknown;
|
||||
};
|
||||
packagePublishTokens: {
|
||||
getByHashInternal: unknown;
|
||||
touchInternal: unknown;
|
||||
};
|
||||
};
|
||||
|
||||
export async function requireApiTokenUser(
|
||||
ctx: ActionCtx,
|
||||
@@ -15,15 +38,26 @@ export async function requireApiTokenUser(
|
||||
if (!token) throw new ConvexError("Unauthorized");
|
||||
|
||||
const tokenHash = await hashToken(token);
|
||||
const apiToken = await ctx.runQuery(internal.tokens.getByHashInternal, { tokenHash });
|
||||
const apiToken = (await ctx.runQuery(
|
||||
internalRefs.tokens.getByHashInternal as never,
|
||||
{
|
||||
tokenHash,
|
||||
} as never,
|
||||
)) as ApiTokenDoc | null;
|
||||
if (!apiToken || apiToken.revokedAt) throw new ConvexError("Unauthorized");
|
||||
|
||||
const user = await ctx.runQuery(internal.tokens.getUserForTokenInternal, {
|
||||
tokenId: apiToken._id,
|
||||
});
|
||||
const user = (await ctx.runQuery(
|
||||
internalRefs.tokens.getUserForTokenInternal as never,
|
||||
{
|
||||
tokenId: apiToken._id,
|
||||
} as never,
|
||||
)) as Doc<"users"> | null;
|
||||
if (!user || user.deletedAt || user.deactivatedAt) throw new ConvexError("Unauthorized");
|
||||
|
||||
await ctx.runMutation(internal.tokens.touchInternal, { tokenId: apiToken._id });
|
||||
await ctx.runMutation(
|
||||
internalRefs.tokens.touchInternal as never,
|
||||
{ tokenId: apiToken._id } as never,
|
||||
);
|
||||
return { user, userId: user._id };
|
||||
}
|
||||
|
||||
@@ -36,18 +70,55 @@ export async function getOptionalApiTokenUserId(
|
||||
if (!token) return null;
|
||||
|
||||
const tokenHash = await hashToken(token);
|
||||
const apiToken = await ctx.runQuery(internal.tokens.getByHashInternal, { tokenHash });
|
||||
const apiToken = (await ctx.runQuery(
|
||||
internalRefs.tokens.getByHashInternal as never,
|
||||
{
|
||||
tokenHash,
|
||||
} as never,
|
||||
)) as ApiTokenDoc | null;
|
||||
if (!apiToken || apiToken.revokedAt) return null;
|
||||
|
||||
const user = await ctx.runQuery(internal.tokens.getUserForTokenInternal, {
|
||||
tokenId: apiToken._id,
|
||||
});
|
||||
const user = (await ctx.runQuery(
|
||||
internalRefs.tokens.getUserForTokenInternal as never,
|
||||
{
|
||||
tokenId: apiToken._id,
|
||||
} as never,
|
||||
)) as Doc<"users"> | null;
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return null;
|
||||
|
||||
return user._id;
|
||||
}
|
||||
|
||||
function parseBearerToken(header: string | null) {
|
||||
export async function requirePackagePublishAuth(
|
||||
ctx: ActionCtx,
|
||||
request: Request,
|
||||
): Promise<UserPackagePublishAuthResult | PackagePublishTokenAuthResult> {
|
||||
const header = request.headers.get("authorization") ?? request.headers.get("Authorization");
|
||||
const token = parseBearerToken(header);
|
||||
if (!token) throw new ConvexError("Unauthorized");
|
||||
|
||||
const tokenHash = await hashToken(token);
|
||||
const publishToken = (await ctx.runQuery(
|
||||
internalRefs.packagePublishTokens.getByHashInternal as never,
|
||||
{
|
||||
tokenHash,
|
||||
} as never,
|
||||
)) as PackagePublishTokenDoc | null;
|
||||
if (publishToken && !publishToken.revokedAt && publishToken.expiresAt > Date.now()) {
|
||||
await ctx.runMutation(
|
||||
internalRefs.packagePublishTokens.touchInternal as never,
|
||||
{
|
||||
tokenId: publishToken._id,
|
||||
} as never,
|
||||
);
|
||||
return { kind: "github-actions", publishToken };
|
||||
}
|
||||
|
||||
const auth = await requireApiTokenUser(ctx, request);
|
||||
return { kind: "user", user: auth.user, userId: auth.userId };
|
||||
}
|
||||
|
||||
export function parseBearerToken(header: string | null) {
|
||||
if (!header) return null;
|
||||
const trimmed = header.trim();
|
||||
if (!trimmed.toLowerCase().startsWith("bearer ")) return null;
|
||||
|
||||
@@ -0,0 +1,334 @@
|
||||
/* @vitest-environment node */
|
||||
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
extractWorkflowFilenameFromWorkflowRef,
|
||||
verifyGitHubActionsTrustedPublishJwt,
|
||||
type TrustedGitHubActionsPublisher,
|
||||
} from "./githubActionsOidc";
|
||||
|
||||
const trustedPublisher: TrustedGitHubActionsPublisher = {
|
||||
repository: "openclaw/openclaw",
|
||||
repositoryId: "123456",
|
||||
repositoryOwner: "openclaw",
|
||||
repositoryOwnerId: "7890",
|
||||
workflowFilename: "plugin-clawhub-release.yml",
|
||||
environment: "clawhub-plugin-release",
|
||||
};
|
||||
const trustedPublisherWithoutEnvironment: TrustedGitHubActionsPublisher = {
|
||||
...trustedPublisher,
|
||||
environment: undefined,
|
||||
};
|
||||
const signingKeyPairPromise = crypto.subtle.generateKey(
|
||||
{
|
||||
name: "RSASSA-PKCS1-v1_5",
|
||||
modulusLength: 2048,
|
||||
publicExponent: new Uint8Array([1, 0, 1]),
|
||||
hash: "SHA-256",
|
||||
},
|
||||
true,
|
||||
["sign", "verify"],
|
||||
);
|
||||
|
||||
describe("extractWorkflowFilenameFromWorkflowRef", () => {
|
||||
it("extracts the workflow filename from workflow_ref", () => {
|
||||
expect(
|
||||
extractWorkflowFilenameFromWorkflowRef(
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
"openclaw/openclaw",
|
||||
),
|
||||
).toBe("plugin-clawhub-release.yml");
|
||||
});
|
||||
});
|
||||
|
||||
describe("verifyGitHubActionsTrustedPublishJwt", () => {
|
||||
it("accepts a valid GitHub Actions token", async () => {
|
||||
const { token, jwks } = await createSignedToken({
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
environment: trustedPublisher.environment,
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
ref_type: "branch",
|
||||
actor: "onur",
|
||||
actor_id: "42",
|
||||
run_id: "100",
|
||||
run_attempt: "2",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(Date.now() / 1000) + 300,
|
||||
iat: Math.floor(Date.now() / 1000) - 5,
|
||||
});
|
||||
|
||||
const identity = await verifyGitHubActionsTrustedPublishJwt(token, trustedPublisher, {
|
||||
fetchImpl: async () =>
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
});
|
||||
|
||||
expect(identity).toMatchObject({
|
||||
repository: trustedPublisher.repository,
|
||||
repositoryId: trustedPublisher.repositoryId,
|
||||
repositoryOwner: trustedPublisher.repositoryOwner,
|
||||
repositoryOwnerId: trustedPublisher.repositoryOwnerId,
|
||||
workflowFilename: trustedPublisher.workflowFilename,
|
||||
environment: trustedPublisher.environment,
|
||||
runId: "100",
|
||||
runAttempt: "2",
|
||||
sha: "deadbeef",
|
||||
});
|
||||
});
|
||||
|
||||
it("accepts a valid GitHub Actions token when no environment is pinned", async () => {
|
||||
const { token, jwks } = await createSignedToken({
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
ref_type: "branch",
|
||||
actor: "onur",
|
||||
actor_id: "42",
|
||||
run_id: "100",
|
||||
run_attempt: "2",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(Date.now() / 1000) + 300,
|
||||
iat: Math.floor(Date.now() / 1000) - 5,
|
||||
});
|
||||
|
||||
const identity = await verifyGitHubActionsTrustedPublishJwt(
|
||||
token,
|
||||
trustedPublisherWithoutEnvironment,
|
||||
{
|
||||
fetchImpl: async () =>
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
},
|
||||
);
|
||||
|
||||
expect(identity).toMatchObject({
|
||||
repository: trustedPublisher.repository,
|
||||
repositoryId: trustedPublisher.repositoryId,
|
||||
repositoryOwner: trustedPublisher.repositoryOwner,
|
||||
repositoryOwnerId: trustedPublisher.repositoryOwnerId,
|
||||
workflowFilename: trustedPublisher.workflowFilename,
|
||||
runId: "100",
|
||||
runAttempt: "2",
|
||||
sha: "deadbeef",
|
||||
});
|
||||
expect(identity.environment).toBeUndefined();
|
||||
});
|
||||
|
||||
it("rejects reusable workflow tokens", async () => {
|
||||
const { token, jwks } = await createSignedToken({
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
job_workflow_ref:
|
||||
"openclaw/shared/.github/workflows/reusable-plugin-release.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
environment: trustedPublisher.environment,
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
run_id: "100",
|
||||
run_attempt: "1",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(Date.now() / 1000) + 300,
|
||||
iat: Math.floor(Date.now() / 1000) - 5,
|
||||
});
|
||||
|
||||
await expect(
|
||||
verifyGitHubActionsTrustedPublishJwt(token, trustedPublisher, {
|
||||
fetchImpl: async () =>
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
}),
|
||||
).rejects.toThrow("Only the official ClawHub reusable workflow is supported");
|
||||
});
|
||||
|
||||
it("accepts the official ClawHub reusable workflow", async () => {
|
||||
const { token, jwks } = await createSignedToken({
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
job_workflow_ref: "openclaw/clawhub/.github/workflows/package-publish.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
environment: trustedPublisher.environment,
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
run_id: "100",
|
||||
run_attempt: "1",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(Date.now() / 1000) + 300,
|
||||
iat: Math.floor(Date.now() / 1000) - 5,
|
||||
});
|
||||
|
||||
await expect(
|
||||
verifyGitHubActionsTrustedPublishJwt(token, trustedPublisher, {
|
||||
fetchImpl: async () =>
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
}),
|
||||
).resolves.toMatchObject({
|
||||
repository: trustedPublisher.repository,
|
||||
workflowFilename: trustedPublisher.workflowFilename,
|
||||
jobWorkflowRef: "openclaw/clawhub/.github/workflows/package-publish.yml@refs/heads/main",
|
||||
});
|
||||
});
|
||||
|
||||
it("rejects environment mismatches", async () => {
|
||||
const { token, jwks } = await createSignedToken({
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
environment: "other-environment",
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
run_id: "100",
|
||||
run_attempt: "1",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(Date.now() / 1000) + 300,
|
||||
iat: Math.floor(Date.now() / 1000) - 5,
|
||||
});
|
||||
|
||||
await expect(
|
||||
verifyGitHubActionsTrustedPublishJwt(token, trustedPublisher, {
|
||||
fetchImpl: async () =>
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
}),
|
||||
).rejects.toThrow("GitHub OIDC environment mismatch");
|
||||
});
|
||||
|
||||
it("refreshes JWKS on signing-key cache misses", async () => {
|
||||
const now = Date.now() + 10 * 60_000;
|
||||
const { token, jwks } = await createSignedToken(
|
||||
{
|
||||
repository: trustedPublisher.repository,
|
||||
repository_id: trustedPublisher.repositoryId,
|
||||
repository_owner: trustedPublisher.repositoryOwner,
|
||||
repository_owner_id: trustedPublisher.repositoryOwnerId,
|
||||
workflow_ref:
|
||||
"openclaw/openclaw/.github/workflows/plugin-clawhub-release.yml@refs/heads/main",
|
||||
runner_environment: "github-hosted",
|
||||
environment: trustedPublisher.environment,
|
||||
event_name: "workflow_dispatch",
|
||||
workflow: "Plugin ClawHub Release",
|
||||
sha: "deadbeef",
|
||||
ref: "refs/heads/main",
|
||||
ref_type: "branch",
|
||||
actor: "onur",
|
||||
actor_id: "42",
|
||||
run_id: "100",
|
||||
run_attempt: "2",
|
||||
iss: "https://token.actions.githubusercontent.com",
|
||||
aud: "clawhub",
|
||||
exp: Math.floor(now / 1000) + 300,
|
||||
iat: Math.floor(now / 1000) - 5,
|
||||
},
|
||||
"rotated-key",
|
||||
);
|
||||
const staleJwk = { ...jwks, kid: "stale-key" };
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({ keys: [staleJwk] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
)
|
||||
.mockResolvedValueOnce(
|
||||
new Response(JSON.stringify({ keys: [jwks] }), {
|
||||
status: 200,
|
||||
headers: { "Content-Type": "application/json" },
|
||||
}),
|
||||
);
|
||||
|
||||
await expect(
|
||||
verifyGitHubActionsTrustedPublishJwt(token, trustedPublisher, {
|
||||
fetchImpl: fetchMock,
|
||||
now: () => now,
|
||||
}),
|
||||
).resolves.toMatchObject({
|
||||
repository: trustedPublisher.repository,
|
||||
workflowFilename: trustedPublisher.workflowFilename,
|
||||
});
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
});
|
||||
|
||||
async function createSignedToken(payload: Record<string, unknown>, kid = "test-key") {
|
||||
const keyPair = await signingKeyPairPromise;
|
||||
const header = { alg: "RS256", kid, typ: "JWT" };
|
||||
const encodedHeader = base64UrlEncodeJson(header);
|
||||
const encodedPayload = base64UrlEncodeJson(payload);
|
||||
const signingInput = `${encodedHeader}.${encodedPayload}`;
|
||||
const signature = new Uint8Array(
|
||||
await crypto.subtle.sign(
|
||||
"RSASSA-PKCS1-v1_5",
|
||||
keyPair.privateKey,
|
||||
new TextEncoder().encode(signingInput),
|
||||
),
|
||||
);
|
||||
const publicJwk = (await crypto.subtle.exportKey("jwk", keyPair.publicKey)) as JsonWebKey & {
|
||||
kid?: string;
|
||||
};
|
||||
publicJwk.kid = kid;
|
||||
return {
|
||||
token: `${signingInput}.${base64UrlEncodeBytes(signature)}`,
|
||||
jwks: publicJwk,
|
||||
};
|
||||
}
|
||||
|
||||
function base64UrlEncodeJson(value: unknown) {
|
||||
return base64UrlEncodeBytes(new TextEncoder().encode(JSON.stringify(value)));
|
||||
}
|
||||
|
||||
function base64UrlEncodeBytes(bytes: Uint8Array) {
|
||||
let binary = "";
|
||||
for (const byte of bytes) binary += String.fromCharCode(byte);
|
||||
return btoa(binary).replace(/\+/g, "-").replace(/\//g, "_").replace(/=+$/g, "");
|
||||
}
|
||||
@@ -0,0 +1,394 @@
|
||||
type JwtHeader = {
|
||||
alg?: unknown;
|
||||
kid?: unknown;
|
||||
typ?: unknown;
|
||||
};
|
||||
|
||||
type JwtPayload = Record<string, unknown>;
|
||||
|
||||
type JwkSet = {
|
||||
keys?: Array<JsonWebKey & { kid?: string; alg?: string; use?: string; kty?: string }>;
|
||||
};
|
||||
|
||||
export type TrustedGitHubActionsPublisher = {
|
||||
repository: string;
|
||||
repositoryId: string;
|
||||
repositoryOwner: string;
|
||||
repositoryOwnerId: string;
|
||||
workflowFilename: string;
|
||||
environment?: string;
|
||||
};
|
||||
|
||||
export type VerifiedGitHubActionsIdentity = {
|
||||
repository: string;
|
||||
repositoryId: string;
|
||||
repositoryOwner: string;
|
||||
repositoryOwnerId: string;
|
||||
workflowFilename: string;
|
||||
workflowName: string;
|
||||
workflowRef: string;
|
||||
jobWorkflowRef?: string;
|
||||
environment?: string;
|
||||
runnerEnvironment: string;
|
||||
eventName: string;
|
||||
sha: string;
|
||||
ref: string;
|
||||
refType?: string;
|
||||
actor?: string;
|
||||
actorId?: string;
|
||||
runId: string;
|
||||
runAttempt: string;
|
||||
};
|
||||
|
||||
type VerifyGitHubActionsOidcOptions = {
|
||||
fetchImpl?: typeof fetch;
|
||||
now?: () => number;
|
||||
};
|
||||
|
||||
type GitHubRepositoryIdentity = {
|
||||
repository: string;
|
||||
repositoryId: string;
|
||||
repositoryOwner: string;
|
||||
repositoryOwnerId: string;
|
||||
};
|
||||
|
||||
type ParsedWorkflowRef = {
|
||||
repository: string;
|
||||
workflowFilename: string;
|
||||
};
|
||||
|
||||
const GITHUB_ACTIONS_ISSUER = "https://token.actions.githubusercontent.com";
|
||||
const GITHUB_ACTIONS_JWKS_URL = `${GITHUB_ACTIONS_ISSUER}/.well-known/jwks`;
|
||||
const TRUSTED_AUDIENCE = "clawhub";
|
||||
const CLOCK_SKEW_MS = 60_000;
|
||||
const JWKS_CACHE_TTL_MS = 5 * 60_000;
|
||||
const OFFICIAL_REUSABLE_WORKFLOW_REPOSITORY = "openclaw/clawhub";
|
||||
const OFFICIAL_REUSABLE_WORKFLOW_FILENAME = "package-publish.yml";
|
||||
|
||||
let cachedJwks: { value: JwkSet; fetchedAt: number } | null = null;
|
||||
|
||||
export async function verifyGitHubActionsTrustedPublishJwt(
|
||||
jwt: string,
|
||||
trustedPublisher: TrustedGitHubActionsPublisher,
|
||||
options: VerifyGitHubActionsOidcOptions = {},
|
||||
): Promise<VerifiedGitHubActionsIdentity> {
|
||||
const fetchImpl = options.fetchImpl ?? fetch;
|
||||
const now = (options.now ?? Date.now)();
|
||||
const { signingInput, signature, header, payload } = decodeJwt(jwt);
|
||||
|
||||
if (header.alg !== "RS256") {
|
||||
throw new Error(
|
||||
`Unsupported GitHub OIDC signing algorithm: ${formatClaimValue(header.alg ?? "<missing>")}`,
|
||||
);
|
||||
}
|
||||
|
||||
const keyId = requireString(header.kid, "kid");
|
||||
let jwks = await fetchGitHubActionsJwks(fetchImpl, now);
|
||||
let jwk = jwks.keys?.find((entry) => entry.kid === keyId);
|
||||
if (!jwk) {
|
||||
jwks = await fetchGitHubActionsJwks(fetchImpl, now, true);
|
||||
jwk = jwks.keys?.find((entry) => entry.kid === keyId);
|
||||
}
|
||||
if (!jwk) throw new Error(`Unknown GitHub OIDC signing key: ${keyId}`);
|
||||
|
||||
const key = await crypto.subtle.importKey(
|
||||
"jwk",
|
||||
jwk,
|
||||
{ name: "RSASSA-PKCS1-v1_5", hash: "SHA-256" },
|
||||
false,
|
||||
["verify"],
|
||||
);
|
||||
const verified = await crypto.subtle.verify(
|
||||
"RSASSA-PKCS1-v1_5",
|
||||
key,
|
||||
signature,
|
||||
new TextEncoder().encode(signingInput),
|
||||
);
|
||||
if (!verified) throw new Error("Invalid GitHub OIDC signature");
|
||||
|
||||
const issuer = requireString(payload.iss, "iss");
|
||||
if (issuer !== GITHUB_ACTIONS_ISSUER) {
|
||||
throw new Error(`Unexpected GitHub OIDC issuer: ${issuer}`);
|
||||
}
|
||||
if (!claimContainsAudience(payload.aud, TRUSTED_AUDIENCE)) {
|
||||
throw new Error(`Unexpected GitHub OIDC audience: ${formatAudience(payload.aud)}`);
|
||||
}
|
||||
assertTokenTimeWindow(payload, now);
|
||||
|
||||
const repository = requireString(payload.repository, "repository");
|
||||
const repositoryId = requireClaimString(payload.repository_id, "repository_id");
|
||||
const repositoryOwner = requireString(payload.repository_owner, "repository_owner");
|
||||
const repositoryOwnerId = requireClaimString(payload.repository_owner_id, "repository_owner_id");
|
||||
const workflowRef = requireString(payload.workflow_ref, "workflow_ref");
|
||||
const workflow = parseWorkflowRef(workflowRef, repository);
|
||||
const jobWorkflowRef = optionalString(payload.job_workflow_ref);
|
||||
const runnerEnvironment = requireString(payload.runner_environment, "runner_environment");
|
||||
const environment = optionalString(payload.environment);
|
||||
const eventName = requireString(payload.event_name, "event_name");
|
||||
const workflowName = requireString(payload.workflow, "workflow");
|
||||
const sha = requireString(payload.sha, "sha");
|
||||
const ref = requireString(payload.ref, "ref");
|
||||
const runId = requireClaimString(payload.run_id, "run_id");
|
||||
const runAttempt = requireClaimString(payload.run_attempt, "run_attempt");
|
||||
const refType = optionalString(payload.ref_type);
|
||||
const actor = optionalString(payload.actor);
|
||||
const actorId = optionalStringValue(payload.actor_id);
|
||||
|
||||
if (repository !== trustedPublisher.repository) {
|
||||
throw new Error(
|
||||
`GitHub OIDC repository mismatch: expected ${trustedPublisher.repository}, got ${repository}`,
|
||||
);
|
||||
}
|
||||
if (repositoryId !== trustedPublisher.repositoryId) {
|
||||
throw new Error(
|
||||
`GitHub OIDC repository_id mismatch: expected ${trustedPublisher.repositoryId}, got ${repositoryId}`,
|
||||
);
|
||||
}
|
||||
if (repositoryOwner !== trustedPublisher.repositoryOwner) {
|
||||
throw new Error(
|
||||
`GitHub OIDC repository_owner mismatch: expected ${trustedPublisher.repositoryOwner}, got ${repositoryOwner}`,
|
||||
);
|
||||
}
|
||||
if (repositoryOwnerId !== trustedPublisher.repositoryOwnerId) {
|
||||
throw new Error(
|
||||
`GitHub OIDC repository_owner_id mismatch: expected ${trustedPublisher.repositoryOwnerId}, got ${repositoryOwnerId}`,
|
||||
);
|
||||
}
|
||||
if (workflow.workflowFilename !== trustedPublisher.workflowFilename) {
|
||||
throw new Error(
|
||||
`GitHub OIDC workflow mismatch: expected ${trustedPublisher.workflowFilename}, got ${workflow.workflowFilename}`,
|
||||
);
|
||||
}
|
||||
if (jobWorkflowRef) {
|
||||
const reusableWorkflow = parseWorkflowRef(jobWorkflowRef);
|
||||
const usesOfficialReusableWorkflow =
|
||||
reusableWorkflow.repository === OFFICIAL_REUSABLE_WORKFLOW_REPOSITORY &&
|
||||
reusableWorkflow.workflowFilename === OFFICIAL_REUSABLE_WORKFLOW_FILENAME;
|
||||
if (!usesOfficialReusableWorkflow) {
|
||||
throw new Error(
|
||||
"Only the official ClawHub reusable workflow is supported for trusted publishing",
|
||||
);
|
||||
}
|
||||
}
|
||||
if (runnerEnvironment !== "github-hosted") {
|
||||
throw new Error(
|
||||
`Only GitHub-hosted runners may mint trusted publish tokens, got ${runnerEnvironment}`,
|
||||
);
|
||||
}
|
||||
// v1 keeps secretless publishing behind a manual entry point. Environment
|
||||
// pinning is optional, but if configured it must match exactly.
|
||||
if (eventName !== "workflow_dispatch") {
|
||||
throw new Error(`Trusted publishing requires workflow_dispatch, got ${eventName}`);
|
||||
}
|
||||
if (trustedPublisher.environment && environment !== trustedPublisher.environment) {
|
||||
throw new Error(
|
||||
`GitHub OIDC environment mismatch: expected ${trustedPublisher.environment}, got ${formatClaimValue(environment ?? "<missing>")}`,
|
||||
);
|
||||
}
|
||||
|
||||
return {
|
||||
repository,
|
||||
repositoryId,
|
||||
repositoryOwner,
|
||||
repositoryOwnerId,
|
||||
workflowFilename: workflow.workflowFilename,
|
||||
workflowName,
|
||||
workflowRef,
|
||||
...(jobWorkflowRef ? { jobWorkflowRef } : {}),
|
||||
...(environment ? { environment } : {}),
|
||||
runnerEnvironment,
|
||||
eventName,
|
||||
sha,
|
||||
ref,
|
||||
...(refType ? { refType } : {}),
|
||||
...(actor ? { actor } : {}),
|
||||
...(actorId ? { actorId } : {}),
|
||||
runId,
|
||||
runAttempt,
|
||||
};
|
||||
}
|
||||
|
||||
export async function fetchGitHubRepositoryIdentity(
|
||||
repository: string,
|
||||
fetchImpl: typeof fetch = fetch,
|
||||
): Promise<GitHubRepositoryIdentity> {
|
||||
const normalizedRepository = normalizeGitHubRepository(repository);
|
||||
if (!normalizedRepository) {
|
||||
throw new Error(`Invalid GitHub repository: ${repository}`);
|
||||
}
|
||||
const response = await fetchImpl(`https://api.github.com/repos/${normalizedRepository}`, {
|
||||
headers: {
|
||||
Accept: "application/vnd.github+json",
|
||||
"User-Agent": "clawhub/package-trusted-publisher",
|
||||
},
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(
|
||||
`GitHub repository lookup failed for ${normalizedRepository}: ${response.status}`,
|
||||
);
|
||||
}
|
||||
const body = (await response.json()) as {
|
||||
id?: unknown;
|
||||
full_name?: unknown;
|
||||
owner?: { login?: unknown; id?: unknown };
|
||||
};
|
||||
const resolvedRepository = requireString(body.full_name, "full_name");
|
||||
const ownerLogin = requireString(body.owner?.login, "owner.login");
|
||||
return {
|
||||
repository: resolvedRepository,
|
||||
repositoryId: requireClaimString(body.id, "id"),
|
||||
repositoryOwner: ownerLogin,
|
||||
repositoryOwnerId: requireClaimString(body.owner?.id, "owner.id"),
|
||||
};
|
||||
}
|
||||
|
||||
export function normalizeGitHubRepository(repository: string) {
|
||||
const trimmed = repository
|
||||
.trim()
|
||||
.replace(/^https?:\/\/github\.com\//i, "")
|
||||
.replace(/\.git$/i, "");
|
||||
const match = /^([A-Za-z0-9_.-]+)\/([A-Za-z0-9_.-]+)$/.exec(trimmed);
|
||||
if (!match) return null;
|
||||
return `${match[1]}/${match[2]}`;
|
||||
}
|
||||
|
||||
export function extractWorkflowFilenameFromWorkflowRef(
|
||||
workflowRef: string,
|
||||
expectedRepository?: string,
|
||||
) {
|
||||
return parseWorkflowRef(workflowRef, expectedRepository).workflowFilename;
|
||||
}
|
||||
|
||||
function parseWorkflowRef(workflowRef: string, expectedRepository?: string): ParsedWorkflowRef {
|
||||
const match = /^([^/]+\/[^/]+)\/\.github\/workflows\/([^@/]+)@.+$/.exec(workflowRef.trim());
|
||||
if (!match?.[1] || !match[2]) {
|
||||
throw new Error(`Invalid GitHub workflow_ref claim: ${workflowRef}`);
|
||||
}
|
||||
if (expectedRepository && match[1] !== expectedRepository) {
|
||||
throw new Error(
|
||||
`GitHub workflow_ref repository mismatch: expected ${expectedRepository}, got ${match[1]}`,
|
||||
);
|
||||
}
|
||||
return {
|
||||
repository: match[1],
|
||||
workflowFilename: match[2],
|
||||
};
|
||||
}
|
||||
|
||||
function decodeJwt(jwt: string) {
|
||||
const parts = jwt.trim().split(".");
|
||||
if (parts.length !== 3) throw new Error("Invalid GitHub OIDC token format");
|
||||
const [encodedHeader, encodedPayload, encodedSignature] = parts;
|
||||
const header = parseJsonSegment<JwtHeader>(encodedHeader, "header");
|
||||
const payload = parseJsonSegment<JwtPayload>(encodedPayload, "payload");
|
||||
return {
|
||||
header,
|
||||
payload,
|
||||
signingInput: `${encodedHeader}.${encodedPayload}`,
|
||||
signature: base64UrlToBytes(encodedSignature),
|
||||
};
|
||||
}
|
||||
|
||||
function parseJsonSegment<T>(segment: string, label: string) {
|
||||
try {
|
||||
return JSON.parse(new TextDecoder().decode(base64UrlToBytes(segment))) as T;
|
||||
} catch {
|
||||
throw new Error(`Invalid GitHub OIDC ${label}`);
|
||||
}
|
||||
}
|
||||
|
||||
function base64UrlToBytes(value: string) {
|
||||
const normalized = value.replace(/-/g, "+").replace(/_/g, "/");
|
||||
const padded = normalized.padEnd(Math.ceil(normalized.length / 4) * 4, "=");
|
||||
const binary = atob(padded);
|
||||
const bytes = new Uint8Array(binary.length);
|
||||
for (let index = 0; index < binary.length; index += 1) {
|
||||
bytes[index] = binary.charCodeAt(index);
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
async function fetchGitHubActionsJwks(fetchImpl: typeof fetch, now: number, forceRefresh = false) {
|
||||
if (!forceRefresh && cachedJwks && now - cachedJwks.fetchedAt < JWKS_CACHE_TTL_MS) {
|
||||
return cachedJwks.value;
|
||||
}
|
||||
const response = await fetchImpl(GITHUB_ACTIONS_JWKS_URL, {
|
||||
headers: {
|
||||
Accept: "application/json",
|
||||
"User-Agent": "clawhub/github-actions-oidc",
|
||||
},
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Failed to fetch GitHub OIDC JWKS: ${response.status}`);
|
||||
}
|
||||
const jwks = (await response.json()) as JwkSet;
|
||||
cachedJwks = { value: jwks, fetchedAt: now };
|
||||
return jwks;
|
||||
}
|
||||
|
||||
function claimContainsAudience(audience: unknown, expected: string) {
|
||||
if (typeof audience === "string") return audience === expected;
|
||||
if (!Array.isArray(audience)) return false;
|
||||
return audience.includes(expected);
|
||||
}
|
||||
|
||||
function formatAudience(audience: unknown) {
|
||||
if (typeof audience === "string") return audience;
|
||||
if (Array.isArray(audience)) return audience.join(", ");
|
||||
return formatClaimValue(audience ?? "<missing>");
|
||||
}
|
||||
|
||||
function formatClaimValue(value: unknown) {
|
||||
if (typeof value === "string" || typeof value === "number" || typeof value === "boolean") {
|
||||
return String(value);
|
||||
}
|
||||
return JSON.stringify(value);
|
||||
}
|
||||
|
||||
function assertTokenTimeWindow(payload: JwtPayload, now: number) {
|
||||
const expiresAt = requireNumericClaim(payload.exp, "exp") * 1000;
|
||||
if (now - CLOCK_SKEW_MS >= expiresAt) {
|
||||
throw new Error("GitHub OIDC token has expired");
|
||||
}
|
||||
const notBefore =
|
||||
payload.nbf === undefined ? undefined : requireNumericClaim(payload.nbf, "nbf") * 1000;
|
||||
if (typeof notBefore === "number" && now + CLOCK_SKEW_MS < notBefore) {
|
||||
throw new Error("GitHub OIDC token is not active yet");
|
||||
}
|
||||
}
|
||||
|
||||
function requireClaimString(value: unknown, label: string) {
|
||||
const normalized = optionalStringValue(value);
|
||||
if (!normalized) throw new Error(`Missing GitHub OIDC claim: ${label}`);
|
||||
return normalized;
|
||||
}
|
||||
|
||||
function requireString(value: unknown, label: string) {
|
||||
if (typeof value !== "string" || !value.trim()) {
|
||||
throw new Error(`Missing GitHub OIDC claim: ${label}`);
|
||||
}
|
||||
return value;
|
||||
}
|
||||
|
||||
function optionalString(value: unknown) {
|
||||
return typeof value === "string" && value.trim() ? value : undefined;
|
||||
}
|
||||
|
||||
function optionalStringValue(value: unknown) {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return String(value);
|
||||
return optionalString(value);
|
||||
}
|
||||
|
||||
function requireNumericClaim(value: unknown, label: string) {
|
||||
if (typeof value === "number" && Number.isFinite(value)) return value;
|
||||
if (typeof value === "string" && value.trim()) {
|
||||
const parsed = Number(value);
|
||||
if (Number.isFinite(parsed)) return parsed;
|
||||
}
|
||||
throw new Error(`Missing GitHub OIDC claim: ${label}`);
|
||||
}
|
||||
|
||||
export const __test = {
|
||||
base64UrlToBytes,
|
||||
claimContainsAudience,
|
||||
};
|
||||
@@ -5,9 +5,9 @@ import { corsHeaders, mergeHeaders } from "./httpHeaders";
|
||||
|
||||
const RATE_LIMIT_WINDOW_MS = 60_000;
|
||||
export const RATE_LIMITS = {
|
||||
read: { ip: 120, key: 600 },
|
||||
write: { ip: 30, key: 120 },
|
||||
download: { ip: 20, key: 120 },
|
||||
read: { ip: 180, key: 900 },
|
||||
write: { ip: 45, key: 180 },
|
||||
download: { ip: 30, key: 180 },
|
||||
} as const;
|
||||
|
||||
type RateLimitResult = {
|
||||
|
||||
@@ -92,6 +92,173 @@ describe("moderationEngine", () => {
|
||||
expect(result.status).toBe("suspicious");
|
||||
});
|
||||
|
||||
it("flags raw user placeholders embedded in generated Python source within markdown", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "word-document-organizer",
|
||||
displayName: "Word Document Organizer",
|
||||
summary: "Organize and restyle Word documents",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 512 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Generate a Python helper like this:",
|
||||
"```python",
|
||||
'doc_path = "${document_path}"',
|
||||
'output_path = "${output_path}" if "${output_path}" else doc_path',
|
||||
'template = "${style_template}"',
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).toContain("suspicious.generated_source_template_injection");
|
||||
expect(result.status).toBe("suspicious");
|
||||
});
|
||||
|
||||
it("does not flag ordinary placeholder usage outside generated source assignments", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "api-docs",
|
||||
displayName: "API Docs",
|
||||
summary: "Shows users how to call an API",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 256 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Use this request template:",
|
||||
"```bash",
|
||||
'curl "https://example.com/search?q=${query}"',
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).not.toContain("suspicious.generated_source_template_injection");
|
||||
expect(result.status).toBe("clean");
|
||||
});
|
||||
|
||||
it("flags hardcoded connection_id UUIDs in markdown examples", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "api-gateway",
|
||||
displayName: "API Gateway",
|
||||
summary: "Route API calls through an authenticated gateway",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 256 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Use this payload:",
|
||||
"```json",
|
||||
'{"connection_id": "21fd90f9-5935-43cd-b6c8-bde9d915ca80"}',
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).toContain("suspicious.exposed_resource_identifier");
|
||||
expect(result.status).toBe("suspicious");
|
||||
expect(
|
||||
result.findings.find((finding) => finding.message.includes("connection_id"))?.message,
|
||||
).toContain("connection_id");
|
||||
});
|
||||
|
||||
it("flags hardcoded Google Sheets spreadsheet IDs in markdown examples", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "api-gateway",
|
||||
displayName: "API Gateway",
|
||||
summary: "Route API calls through an authenticated gateway",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 256 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Call the Sheets bridge like this:",
|
||||
"```python",
|
||||
"req = urllib.request.Request('https://gateway.maton.ai/google-sheets/v4/spreadsheets/122BS1sFN2RKL8AOUQjkLdubzOwgqzPT64KfZ2rvYI4M/values/Sheet1!A1:B2')",
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).toContain("suspicious.exposed_resource_identifier");
|
||||
expect(result.status).toBe("suspicious");
|
||||
expect(
|
||||
result.findings.find((finding) => finding.message.includes("spreadsheet ID"))?.message,
|
||||
).toContain("spreadsheet ID");
|
||||
});
|
||||
|
||||
it("does not flag placeholder resource identifiers in markdown examples", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "api-gateway",
|
||||
displayName: "API Gateway",
|
||||
summary: "Route API calls through an authenticated gateway",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 256 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Use placeholders in public docs:",
|
||||
"```json",
|
||||
'{"connection_id": "YOUR_CONNECTION_ID"}',
|
||||
"```",
|
||||
"```python",
|
||||
"req = urllib.request.Request('https://gateway.maton.ai/google-sheets/v4/spreadsheets/YOUR_SPREADSHEET_ID/values/Sheet1!A1:B2')",
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).not.toContain("suspicious.exposed_resource_identifier");
|
||||
expect(result.status).toBe("clean");
|
||||
});
|
||||
|
||||
it("flags a real spreadsheet ID even when a placeholder URL appears first", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "api-gateway",
|
||||
displayName: "API Gateway",
|
||||
summary: "Route API calls through an authenticated gateway",
|
||||
frontmatter: {},
|
||||
metadata: {},
|
||||
files: [{ path: "SKILL.md", size: 512 }],
|
||||
fileContents: [
|
||||
{
|
||||
path: "SKILL.md",
|
||||
content: [
|
||||
"Placeholder example first:",
|
||||
"```python",
|
||||
"req = urllib.request.Request('https://gateway.maton.ai/google-sheets/v4/spreadsheets/YOUR_SPREADSHEET_ID/values/Sheet1!A1:B2')",
|
||||
"```",
|
||||
"Real leaked URL later:",
|
||||
"```python",
|
||||
"req = urllib.request.Request('https://gateway.maton.ai/google-sheets/v4/spreadsheets/122BS1sFN2RKL8AOUQjkLdubzOwgqzPT64KfZ2rvYI4M/values/Sheet1!A1:B2')",
|
||||
"```",
|
||||
].join("\n"),
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(result.reasonCodes).toContain("suspicious.exposed_resource_identifier");
|
||||
expect(
|
||||
result.findings.find((finding) => finding.message.includes("spreadsheet ID"))?.line,
|
||||
).toBe(7);
|
||||
});
|
||||
|
||||
it("blocks obfuscated terminal install payload prompts in markdown", () => {
|
||||
const result = runStaticModerationScan({
|
||||
slug: "evil-installer",
|
||||
|
||||
@@ -50,6 +50,14 @@ const CODE_EXTENSION = /\.(js|ts|mjs|cjs|mts|cts|jsx|tsx|py|sh|bash|zsh|rb|go)$/
|
||||
const STANDARD_PORTS = new Set([80, 443, 8080, 8443, 3000]);
|
||||
const RAW_IP_URL_PATTERN = /https?:\/\/\d{1,3}(?:\.\d{1,3}){3}(?::\d+)?(?:\/|["'])/i;
|
||||
const INSTALL_PACKAGE_PATTERN = /installer-package\s*:\s*https?:\/\/[^\s"'`]+/i;
|
||||
const GENERATED_SOURCE_PLACEHOLDER_PATTERN =
|
||||
/^\s*[A-Za-z_][A-Za-z0-9_]*\s*=.*["']\$\{[A-Za-z_][A-Za-z0-9_-]*\}["']/m;
|
||||
const GENERATED_SOURCE_CONTEXT_PATTERN =
|
||||
/```(?:python|py|javascript|js|typescript|ts|shell|bash|sh)\b|cat\s*(?:>|>>)?\s*[^`\n]*\.(?:py|js|ts|sh)\b|python3?\b|node\b/i;
|
||||
const HARDCODED_CONNECTION_ID_PATTERN =
|
||||
/["']connection_id["']\s*:\s*["'][0-9a-f]{8}-[0-9a-f]{4}-[1-5][0-9a-f]{3}-[89ab][0-9a-f]{3}-[0-9a-f]{12}["']/i;
|
||||
const GOOGLE_SHEETS_SPREADSHEET_URL_PATTERN =
|
||||
/https?:\/\/[^\s"'`]*\/spreadsheets\/([A-Za-z0-9_-]{20,})\/[^\s"'`]*/i;
|
||||
|
||||
function hasMaliciousInstallPrompt(content: string) {
|
||||
const hasTerminalInstruction =
|
||||
@@ -75,6 +83,10 @@ function truncateEvidence(evidence: string, maxLen = 160) {
|
||||
return `${evidence.slice(0, maxLen)}...`;
|
||||
}
|
||||
|
||||
function looksLikePlaceholderIdentifier(identifier: string) {
|
||||
return /^[A-Z0-9_]+$/.test(identifier) || /(your|example|placeholder)/i.test(identifier);
|
||||
}
|
||||
|
||||
function addFinding(
|
||||
findings: ModerationFinding[],
|
||||
finding: Omit<ModerationFinding, "evidence"> & { evidence: string },
|
||||
@@ -92,6 +104,14 @@ function findFirstLine(content: string, pattern: RegExp) {
|
||||
return { line: 1, text: lines[0] ?? "" };
|
||||
}
|
||||
|
||||
function findLineAtIndex(content: string, index: number) {
|
||||
const line = content.slice(0, index).split("\n").length;
|
||||
const lineStart = content.lastIndexOf("\n", Math.max(0, index - 1)) + 1;
|
||||
const nextNewline = content.indexOf("\n", index);
|
||||
const lineEnd = nextNewline === -1 ? content.length : nextNewline;
|
||||
return { line, text: content.slice(lineStart, lineEnd) };
|
||||
}
|
||||
|
||||
function scanCodeFile(path: string, content: string, findings: ModerationFinding[]) {
|
||||
if (!CODE_EXTENSION.test(path)) return;
|
||||
|
||||
@@ -227,6 +247,53 @@ function scanMarkdownFile(path: string, content: string, findings: ModerationFin
|
||||
evidence: match.text,
|
||||
});
|
||||
}
|
||||
|
||||
if (
|
||||
GENERATED_SOURCE_PLACEHOLDER_PATTERN.test(content) &&
|
||||
GENERATED_SOURCE_CONTEXT_PATTERN.test(content)
|
||||
) {
|
||||
const match = findFirstLine(content, GENERATED_SOURCE_PLACEHOLDER_PATTERN);
|
||||
addFinding(findings, {
|
||||
code: REASON_CODES.GENERATED_SOURCE_TEMPLATE,
|
||||
severity: "critical",
|
||||
file: path,
|
||||
line: match.line,
|
||||
message: "User-controlled placeholder is embedded directly into generated source code.",
|
||||
evidence: match.text,
|
||||
});
|
||||
}
|
||||
|
||||
if (HARDCODED_CONNECTION_ID_PATTERN.test(content)) {
|
||||
const match = findFirstLine(content, HARDCODED_CONNECTION_ID_PATTERN);
|
||||
addFinding(findings, {
|
||||
code: REASON_CODES.EXPOSED_RESOURCE_IDENTIFIER,
|
||||
severity: "critical",
|
||||
file: path,
|
||||
line: match.line,
|
||||
message: "Example code exposes a concrete connection_id instead of a placeholder.",
|
||||
evidence: match.text,
|
||||
});
|
||||
}
|
||||
|
||||
const spreadsheetUrlPattern = new RegExp(
|
||||
GOOGLE_SHEETS_SPREADSHEET_URL_PATTERN.source,
|
||||
`${GOOGLE_SHEETS_SPREADSHEET_URL_PATTERN.flags.replaceAll("g", "")}g`,
|
||||
);
|
||||
for (const spreadsheetUrlMatch of content.matchAll(spreadsheetUrlPattern)) {
|
||||
const spreadsheetId = spreadsheetUrlMatch[1];
|
||||
if (!spreadsheetId || looksLikePlaceholderIdentifier(spreadsheetId)) continue;
|
||||
|
||||
const match = findLineAtIndex(content, spreadsheetUrlMatch.index ?? 0);
|
||||
addFinding(findings, {
|
||||
code: REASON_CODES.EXPOSED_RESOURCE_IDENTIFIER,
|
||||
severity: "critical",
|
||||
file: path,
|
||||
line: match.line,
|
||||
message: "Example code exposes a concrete Google Sheets spreadsheet ID instead of a placeholder.",
|
||||
evidence: match.text,
|
||||
});
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
function scanManifestFile(path: string, content: string, findings: ModerationFinding[]) {
|
||||
|
||||
@@ -12,11 +12,13 @@ export type ModerationFinding = {
|
||||
evidence: string;
|
||||
};
|
||||
|
||||
export const MODERATION_ENGINE_VERSION = "v2.2.0";
|
||||
export const MODERATION_ENGINE_VERSION = "v2.4.0";
|
||||
|
||||
export const REASON_CODES = {
|
||||
DANGEROUS_EXEC: "suspicious.dangerous_exec",
|
||||
DYNAMIC_CODE: "suspicious.dynamic_code_execution",
|
||||
GENERATED_SOURCE_TEMPLATE: "suspicious.generated_source_template_injection",
|
||||
EXPOSED_RESOURCE_IDENTIFIER: "suspicious.exposed_resource_identifier",
|
||||
CREDENTIAL_HARVEST: "suspicious.env_credential_access",
|
||||
EXFILTRATION: "suspicious.potential_exfiltration",
|
||||
OBFUSCATED_CODE: "suspicious.obfuscated_code",
|
||||
|
||||
@@ -46,6 +46,7 @@ describe("packageRegistry", () => {
|
||||
|
||||
expect(result.runtimeId).toBe("demo.plugin");
|
||||
expect(result.compatibility?.pluginApiRange).toBe("^1.2.0");
|
||||
expect(result.compatibility?.minGatewayVersion).toBe("2026.3.0");
|
||||
expect(result.capabilities.executesCode).toBe(true);
|
||||
expect(result.capabilities.toolNames).toContain("demoTool");
|
||||
expect(result.verification.tier).toBe("source-linked");
|
||||
@@ -70,38 +71,60 @@ describe("packageRegistry", () => {
|
||||
).toThrow("source repo and commit");
|
||||
});
|
||||
|
||||
it("infers compatibility for legacy openclaw extension manifests", () => {
|
||||
const result = extractCodePluginArtifacts({
|
||||
packageName: "@openclaw/matrix",
|
||||
it("maps legacy minHostVersion to minGatewayVersion instead of pluginApiRange", () => {
|
||||
expect(() =>
|
||||
extractCodePluginArtifacts({
|
||||
packageName: "@openclaw/matrix",
|
||||
packageJson: {
|
||||
name: "@openclaw/matrix",
|
||||
version: "2026.3.13",
|
||||
openclaw: {
|
||||
extensions: ["./index.ts"],
|
||||
install: {
|
||||
npmSpec: "@openclaw/matrix",
|
||||
localPath: "extensions/matrix",
|
||||
defaultChoice: "npm",
|
||||
minHostVersion: "2026.3.13",
|
||||
},
|
||||
},
|
||||
},
|
||||
pluginManifest: {
|
||||
id: "matrix",
|
||||
channels: ["matrix"],
|
||||
configSchema: { type: "object" },
|
||||
},
|
||||
source: {
|
||||
kind: "github",
|
||||
url: "https://github.com/openclaw/openclaw",
|
||||
repo: "openclaw/openclaw",
|
||||
ref: "refs/tags/v2026.3.13",
|
||||
commit: "abc123",
|
||||
path: "extensions/matrix",
|
||||
importedAt: Date.now(),
|
||||
},
|
||||
}),
|
||||
).toThrow("package.json openclaw.compat.pluginApi is required");
|
||||
});
|
||||
|
||||
it("extracts legacy minHostVersion as minGatewayVersion while preserving build metadata", () => {
|
||||
const result = extractBundlePluginArtifacts({
|
||||
packageName: "@openclaw/matrix-bundle",
|
||||
packageJson: {
|
||||
name: "@openclaw/matrix",
|
||||
name: "@openclaw/matrix-bundle",
|
||||
version: "2026.3.13",
|
||||
openclaw: {
|
||||
extensions: ["./index.ts"],
|
||||
install: {
|
||||
npmSpec: "@openclaw/matrix",
|
||||
localPath: "extensions/matrix",
|
||||
defaultChoice: "npm",
|
||||
minHostVersion: "2026.3.13",
|
||||
},
|
||||
},
|
||||
},
|
||||
pluginManifest: {
|
||||
id: "matrix",
|
||||
channels: ["matrix"],
|
||||
configSchema: { type: "object" },
|
||||
},
|
||||
source: {
|
||||
kind: "github",
|
||||
url: "https://github.com/openclaw/openclaw",
|
||||
repo: "openclaw/openclaw",
|
||||
ref: "refs/tags/v2026.3.13",
|
||||
commit: "abc123",
|
||||
path: "extensions/matrix",
|
||||
importedAt: Date.now(),
|
||||
bundleManifest: {
|
||||
hostTargets: ["openclaw"],
|
||||
},
|
||||
});
|
||||
|
||||
expect(result.compatibility?.pluginApiRange).toBe(">=2026.3.13");
|
||||
expect(result.compatibility?.pluginApiRange).toBeUndefined();
|
||||
expect(result.compatibility?.minGatewayVersion).toBe("2026.3.13");
|
||||
expect(result.compatibility?.builtWithOpenClawVersion).toBe("2026.3.13");
|
||||
});
|
||||
|
||||
@@ -116,9 +139,9 @@ describe("packageRegistry", () => {
|
||||
|
||||
it("validates package name consistency and summary extraction", () => {
|
||||
ensurePluginNameMatchesPackage("demo-plugin", { name: "demo-plugin" });
|
||||
expect(() =>
|
||||
ensurePluginNameMatchesPackage("demo-plugin", { name: "other-plugin" }),
|
||||
).toThrow("must match published package name");
|
||||
expect(() => ensurePluginNameMatchesPackage("demo-plugin", { name: "other-plugin" })).toThrow(
|
||||
"must match published package name",
|
||||
);
|
||||
|
||||
expect(
|
||||
summarizePackageForSearch({
|
||||
|
||||
@@ -1,3 +1,7 @@
|
||||
import {
|
||||
listMissingOpenClawExternalCodePluginFieldPaths,
|
||||
normalizeOpenClawExternalPluginCompatibility,
|
||||
} from "clawhub-schema";
|
||||
import type {
|
||||
BundlePublishMetadata,
|
||||
PackageCapabilitySummary,
|
||||
@@ -84,7 +88,10 @@ export function normalizePublishFiles(files: PublishFile[]) {
|
||||
return normalized.map((file) => ({ ...file, path: file.path as string }));
|
||||
}
|
||||
|
||||
export function assertPackageVersion(family: "code-plugin" | "bundle-plugin" | "skill", version: string) {
|
||||
export function assertPackageVersion(
|
||||
family: "code-plugin" | "bundle-plugin" | "skill",
|
||||
version: string,
|
||||
) {
|
||||
const trimmed = version.trim();
|
||||
if (!trimmed) throw new ConvexError("Version required");
|
||||
if (family === "code-plugin" && !semver.valid(trimmed)) {
|
||||
@@ -125,9 +132,15 @@ function parseJsonFile(text: string, label: string): JsonRecord {
|
||||
}
|
||||
}
|
||||
|
||||
function deriveSummary(params: { packageName: string; packageJson?: JsonRecord; readmeText?: string | null }) {
|
||||
function deriveSummary(params: {
|
||||
packageName: string;
|
||||
packageJson?: JsonRecord;
|
||||
readmeText?: string | null;
|
||||
}) {
|
||||
const directDescription =
|
||||
typeof params.packageJson?.description === "string" ? params.packageJson.description.trim() : "";
|
||||
typeof params.packageJson?.description === "string"
|
||||
? params.packageJson.description.trim()
|
||||
: "";
|
||||
if (directDescription) return directDescription;
|
||||
const readme = params.readmeText?.trim() ?? "";
|
||||
if (!readme) return params.packageName;
|
||||
@@ -165,52 +178,10 @@ function buildVerification(source: SourceInfo | undefined): PackageVerificationS
|
||||
};
|
||||
}
|
||||
|
||||
function extractOpenClawBlock(packageJson: JsonRecord | undefined) {
|
||||
if (!packageJson) return {};
|
||||
const openclaw = isRecord(packageJson.openclaw) ? packageJson.openclaw : undefined;
|
||||
return {
|
||||
openclaw,
|
||||
compat: isRecord(openclaw?.compat) ? openclaw.compat : undefined,
|
||||
build: isRecord(openclaw?.build) ? openclaw.build : undefined,
|
||||
};
|
||||
}
|
||||
|
||||
function extractCompatibility(packageJson: JsonRecord | undefined): PackageCompatibility | undefined {
|
||||
const { openclaw, compat, build } = extractOpenClawBlock(packageJson);
|
||||
const install = isRecord(openclaw?.install) ? openclaw.install : undefined;
|
||||
const peerDependencies = isRecord(packageJson?.peerDependencies)
|
||||
? packageJson.peerDependencies
|
||||
: undefined;
|
||||
const version =
|
||||
typeof packageJson?.version === "string" ? packageJson.version.trim() : undefined;
|
||||
const peerOpenClaw =
|
||||
typeof peerDependencies?.openclaw === "string" ? peerDependencies.openclaw.trim() : undefined;
|
||||
const minHostVersion =
|
||||
typeof install?.minHostVersion === "string" ? install.minHostVersion.trim() : undefined;
|
||||
const compatibility: PackageCompatibility = {};
|
||||
if (typeof compat?.pluginApi === "string") {
|
||||
compatibility.pluginApiRange = compat.pluginApi.trim();
|
||||
} else if (peerOpenClaw) {
|
||||
compatibility.pluginApiRange = peerOpenClaw;
|
||||
} else if (minHostVersion) {
|
||||
compatibility.pluginApiRange = minHostVersion;
|
||||
} else if (version) {
|
||||
compatibility.pluginApiRange = `>=${version}`;
|
||||
}
|
||||
if (typeof compat?.minGatewayVersion === "string") {
|
||||
compatibility.minGatewayVersion = compat.minGatewayVersion.trim();
|
||||
} else if (minHostVersion) {
|
||||
compatibility.minGatewayVersion = minHostVersion;
|
||||
}
|
||||
if (typeof build?.openclawVersion === "string") {
|
||||
compatibility.builtWithOpenClawVersion = build.openclawVersion.trim();
|
||||
} else if (version) {
|
||||
compatibility.builtWithOpenClawVersion = version;
|
||||
}
|
||||
if (typeof build?.pluginSdkVersion === "string") {
|
||||
compatibility.pluginSdkVersion = build.pluginSdkVersion.trim();
|
||||
}
|
||||
return Object.keys(compatibility).length > 0 ? compatibility : undefined;
|
||||
function extractCompatibility(
|
||||
packageJson: JsonRecord | undefined,
|
||||
): PackageCompatibility | undefined {
|
||||
return normalizeOpenClawExternalPluginCompatibility(packageJson);
|
||||
}
|
||||
|
||||
export function extractCodePluginArtifacts(params: {
|
||||
@@ -223,7 +194,7 @@ export function extractCodePluginArtifacts(params: {
|
||||
throw new ConvexError("Code plugins must include source repo and commit metadata");
|
||||
}
|
||||
|
||||
const { openclaw } = extractOpenClawBlock(params.packageJson);
|
||||
const openclaw = isRecord(params.packageJson.openclaw) ? params.packageJson.openclaw : undefined;
|
||||
const extensions = normalizeStringList(openclaw?.extensions);
|
||||
if (extensions.length === 0) {
|
||||
throw new ConvexError("package.json must declare openclaw.extensions");
|
||||
@@ -234,11 +205,9 @@ export function extractCodePluginArtifacts(params: {
|
||||
if (!runtimeId) throw new ConvexError("openclaw.plugin.json must declare an id");
|
||||
|
||||
const compatibility = extractCompatibility(params.packageJson);
|
||||
if (!compatibility?.pluginApiRange) {
|
||||
throw new ConvexError("package.json openclaw.compat.pluginApi is required");
|
||||
}
|
||||
if (!compatibility.builtWithOpenClawVersion) {
|
||||
throw new ConvexError("package.json openclaw.build.openclawVersion is required");
|
||||
const missingOpenClawFields = listMissingOpenClawExternalCodePluginFieldPaths(params.packageJson);
|
||||
if (missingOpenClawFields.length > 0) {
|
||||
throw new ConvexError(`package.json ${missingOpenClawFields[0]} is required`);
|
||||
}
|
||||
|
||||
const channels = uniq([
|
||||
@@ -280,7 +249,9 @@ export function extractCodePluginArtifacts(params: {
|
||||
executesCode: true,
|
||||
runtimeId,
|
||||
pluginKind:
|
||||
typeof params.pluginManifest.kind === "string" ? params.pluginManifest.kind.trim() : undefined,
|
||||
typeof params.pluginManifest.kind === "string"
|
||||
? params.pluginManifest.kind.trim()
|
||||
: undefined,
|
||||
channels,
|
||||
providers,
|
||||
hooks,
|
||||
@@ -322,7 +293,7 @@ export function extractBundlePluginArtifacts(params: {
|
||||
bundleMetadata?: BundlePublishMetadata;
|
||||
source?: SourceInfo;
|
||||
}) {
|
||||
const { openclaw } = extractOpenClawBlock(params.packageJson);
|
||||
const openclaw = isRecord(params.packageJson?.openclaw) ? params.packageJson.openclaw : undefined;
|
||||
const manifest = params.bundleManifest;
|
||||
const runtimeId =
|
||||
(typeof manifest?.id === "string" && manifest.id.trim()) ||
|
||||
@@ -376,7 +347,9 @@ export function ensurePluginNameMatchesPackage(packageName: string, packageJson:
|
||||
const normalizedDeclared = normalizePackageName(declaredName);
|
||||
const normalizedExpected = normalizePackageName(packageName);
|
||||
if (normalizedDeclared !== normalizedExpected) {
|
||||
throw new ConvexError(`package.json name must match published package name (${normalizedExpected})`);
|
||||
throw new ConvexError(
|
||||
`package.json name must match published package name (${normalizedExpected})`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -5,6 +5,8 @@ function pick<T extends Record<string, unknown>, K extends keyof T>(obj: T, keys
|
||||
return Object.fromEntries(keys.map((key) => [key, obj[key]])) as Pick<T, K>;
|
||||
}
|
||||
|
||||
type SharedPackageKey = Extract<keyof Doc<"packages">, keyof Doc<"packageSearchDigest">>;
|
||||
|
||||
const SHARED_KEYS = [
|
||||
"name",
|
||||
"normalizedName",
|
||||
@@ -22,7 +24,7 @@ const SHARED_KEYS = [
|
||||
"softDeletedAt",
|
||||
"createdAt",
|
||||
"updatedAt",
|
||||
] as const satisfies readonly (keyof Doc<"packages"> & keyof Doc<"packageSearchDigest">)[];
|
||||
] as const satisfies readonly SharedPackageKey[];
|
||||
|
||||
const CAPABILITY_SHARED_KEYS = [
|
||||
"packageId",
|
||||
@@ -145,10 +147,7 @@ export async function deletePackageSearchDigests(
|
||||
function hasDigestChanged<
|
||||
TExisting extends Record<string, unknown>,
|
||||
TFields extends Record<string, unknown>,
|
||||
>(
|
||||
existing: TExisting,
|
||||
fields: TFields,
|
||||
): boolean {
|
||||
>(existing: TExisting, fields: TFields): boolean {
|
||||
for (const key of Object.keys(fields)) {
|
||||
const oldValue = (existing as Record<string, unknown>)[key];
|
||||
const newValue = (fields as Record<string, unknown>)[key];
|
||||
|
||||
@@ -0,0 +1,50 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
getPackageDownloadSecurityBlock,
|
||||
isPackageBlockedFromPublic,
|
||||
resolvePackageReleaseScanStatus,
|
||||
} from "./packageSecurity";
|
||||
|
||||
describe("packageSecurity", () => {
|
||||
it("treats pending package scans as public", () => {
|
||||
expect(isPackageBlockedFromPublic("pending")).toBe(false);
|
||||
});
|
||||
|
||||
it("allows package downloads while VT is pending", () => {
|
||||
expect(
|
||||
getPackageDownloadSecurityBlock({
|
||||
sha256hash: "a".repeat(64),
|
||||
} as never),
|
||||
).toBeNull();
|
||||
});
|
||||
|
||||
it("still resolves sha256-only releases to pending", () => {
|
||||
expect(
|
||||
resolvePackageReleaseScanStatus({
|
||||
sha256hash: "a".repeat(64),
|
||||
} as never),
|
||||
).toBe("pending");
|
||||
});
|
||||
|
||||
it("still blocks malicious package releases", () => {
|
||||
expect(isPackageBlockedFromPublic("malicious")).toBe(true);
|
||||
expect(
|
||||
getPackageDownloadSecurityBlock({
|
||||
vtAnalysis: { status: "malicious" },
|
||||
} as never),
|
||||
).toEqual(
|
||||
expect.objectContaining({
|
||||
status: 403,
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("treats suspicious static scans as suspicious even when verification is clean", () => {
|
||||
expect(
|
||||
resolvePackageReleaseScanStatus({
|
||||
staticScan: { status: "suspicious" },
|
||||
verification: { scanStatus: "clean" },
|
||||
} as never),
|
||||
).toBe("suspicious");
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,61 @@
|
||||
import type { Doc } from "../_generated/dataModel";
|
||||
|
||||
export type PackageScanStatus = Doc<"packages">["scanStatus"];
|
||||
|
||||
type PackageReleaseSecurityLike = Pick<
|
||||
Doc<"packageReleases">,
|
||||
"sha256hash" | "vtAnalysis" | "verification" | "staticScan"
|
||||
>;
|
||||
|
||||
export function normalizePackageScanStatus(status: string | null | undefined): PackageScanStatus {
|
||||
switch (status?.trim().toLowerCase()) {
|
||||
case "clean":
|
||||
case "suspicious":
|
||||
case "malicious":
|
||||
case "pending":
|
||||
case "not-run":
|
||||
return status.trim().toLowerCase() as PackageScanStatus;
|
||||
default:
|
||||
return undefined;
|
||||
}
|
||||
}
|
||||
|
||||
export function resolvePackageReleaseScanStatus(
|
||||
release: PackageReleaseSecurityLike,
|
||||
): Exclude<PackageScanStatus, undefined> {
|
||||
const staticStatus = normalizePackageScanStatus(release.staticScan?.status);
|
||||
if (staticStatus === "malicious") return "malicious";
|
||||
if (staticStatus === "suspicious") return "suspicious";
|
||||
|
||||
const vtStatus = normalizePackageScanStatus(release.vtAnalysis?.status);
|
||||
if (vtStatus === "malicious") return "malicious";
|
||||
if (vtStatus === "suspicious") return "suspicious";
|
||||
|
||||
const verificationStatus = normalizePackageScanStatus(release.verification?.scanStatus);
|
||||
if (verificationStatus === "malicious") return "malicious";
|
||||
if (verificationStatus === "suspicious") return "suspicious";
|
||||
|
||||
if (vtStatus) return vtStatus;
|
||||
if (verificationStatus && verificationStatus !== "not-run") return verificationStatus;
|
||||
if (release.sha256hash) return "pending";
|
||||
|
||||
return verificationStatus ?? "not-run";
|
||||
}
|
||||
|
||||
export function isPackageBlockedFromPublic(scanStatus: PackageScanStatus) {
|
||||
return scanStatus === "malicious";
|
||||
}
|
||||
|
||||
export function getPackageDownloadSecurityBlock(release: PackageReleaseSecurityLike) {
|
||||
const scanStatus = resolvePackageReleaseScanStatus(release);
|
||||
|
||||
if (scanStatus === "malicious") {
|
||||
return {
|
||||
status: 403,
|
||||
message:
|
||||
"Blocked: this package release has been flagged as malicious and cannot be downloaded.",
|
||||
};
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
@@ -24,6 +24,7 @@ export type PublicSkill = Pick<
|
||||
| "forkOf"
|
||||
| "latestVersionId"
|
||||
| "tags"
|
||||
| "capabilityTags"
|
||||
| "badges"
|
||||
| "stats"
|
||||
| "createdAt"
|
||||
@@ -50,6 +51,7 @@ export type HydratableSkill = Pick<
|
||||
| "latestVersionId"
|
||||
| "latestVersionSummary"
|
||||
| "tags"
|
||||
| "capabilityTags"
|
||||
| "badges"
|
||||
| "stats"
|
||||
| "statsDownloads"
|
||||
@@ -141,6 +143,7 @@ export function toPublicSkill(skill: HydratableSkill | null | undefined): Public
|
||||
forkOf: skill.forkOf,
|
||||
latestVersionId: skill.latestVersionId,
|
||||
tags: skill.tags,
|
||||
capabilityTags: skill.capabilityTags,
|
||||
badges: skill.badges,
|
||||
stats,
|
||||
createdAt: skill.createdAt,
|
||||
|
||||
+59
-25
@@ -18,8 +18,7 @@ function derivePersonalPublisherHandle(user: Doc<"users">) {
|
||||
const emailLocalPart = user.email?.split("@")[0];
|
||||
const userIdSuffix = String(user._id).split(":").pop();
|
||||
return (
|
||||
normalizePublisherHandle(user.handle ?? user.name ?? emailLocalPart ?? userIdSuffix) ??
|
||||
"user"
|
||||
normalizePublisherHandle(user.handle ?? user.name ?? emailLocalPart ?? userIdSuffix) ?? "user"
|
||||
);
|
||||
}
|
||||
|
||||
@@ -27,7 +26,8 @@ function synthesizePersonalPublisher(user: Doc<"users">): Doc<"publishers"> {
|
||||
const handle = derivePersonalPublisherHandle(user);
|
||||
const now = user.updatedAt ?? user.createdAt ?? user._creationTime;
|
||||
return {
|
||||
_id: (user.personalPublisherId ?? (`publishers:${handle}` as Id<"publishers">)) as Id<"publishers">,
|
||||
_id: (user.personalPublisherId ??
|
||||
(`publishers:${handle}` as Id<"publishers">)) as Id<"publishers">,
|
||||
_creationTime: user._creationTime,
|
||||
kind: "user",
|
||||
handle,
|
||||
@@ -43,6 +43,20 @@ function synthesizePersonalPublisher(user: Doc<"users">): Doc<"publishers"> {
|
||||
};
|
||||
}
|
||||
|
||||
export async function getPersonalPublisherForUserOrFallback(ctx: DbCtx, user: Doc<"users">) {
|
||||
if (user.personalPublisherId) {
|
||||
const publisher = await ctx.db.get(user.personalPublisherId);
|
||||
if (isPublisherActive(publisher)) return publisher;
|
||||
}
|
||||
try {
|
||||
const publisher = await getPersonalPublisherForUser(ctx, user._id);
|
||||
if (isPublisherActive(publisher)) return publisher;
|
||||
} catch (error) {
|
||||
if (!isMissingPublisherTableError(error)) throw error;
|
||||
}
|
||||
return synthesizePersonalPublisher(user);
|
||||
}
|
||||
|
||||
export function normalizePublisherHandle(handle: string | undefined | null) {
|
||||
const normalized = handle?.trim().replace(/^@+/, "").toLowerCase();
|
||||
return normalized ? normalized : undefined;
|
||||
@@ -63,10 +77,7 @@ export function isPublisherRoleAllowed(role: PublisherRole, allowed: PublisherRo
|
||||
return allowed.some((candidate) => ranks[role] >= ranks[candidate]);
|
||||
}
|
||||
|
||||
export async function getPublisherByHandle(
|
||||
ctx: DbCtx,
|
||||
handle: string | undefined | null,
|
||||
) {
|
||||
export async function getPublisherByHandle(ctx: DbCtx, handle: string | undefined | null) {
|
||||
const normalized = normalizePublisherHandle(handle);
|
||||
if (!normalized) return null;
|
||||
try {
|
||||
@@ -80,10 +91,42 @@ export async function getPublisherByHandle(
|
||||
}
|
||||
}
|
||||
|
||||
export async function getPersonalPublisherForUser(
|
||||
export async function getUserByHandleOrPersonalPublisher(
|
||||
ctx: DbCtx,
|
||||
userId: Id<"users">,
|
||||
handle: string | undefined | null,
|
||||
) {
|
||||
const normalized = normalizePublisherHandle(handle);
|
||||
if (!normalized) return null;
|
||||
|
||||
const user = await ctx.db
|
||||
.query("users")
|
||||
.withIndex("handle", (q) => q.eq("handle", normalized))
|
||||
.unique();
|
||||
if (user) return user;
|
||||
|
||||
const publisher = await getPublisherByHandle(ctx, normalized);
|
||||
if (
|
||||
!publisher ||
|
||||
!isPublisherActive(publisher) ||
|
||||
publisher.kind !== "user" ||
|
||||
!publisher.linkedUserId
|
||||
) {
|
||||
return null;
|
||||
}
|
||||
|
||||
return await ctx.db.get(publisher.linkedUserId);
|
||||
}
|
||||
|
||||
export async function getActiveUserByHandleOrPersonalPublisher(
|
||||
ctx: DbCtx,
|
||||
handle: string | undefined | null,
|
||||
) {
|
||||
const user = await getUserByHandleOrPersonalPublisher(ctx, handle);
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return null;
|
||||
return user;
|
||||
}
|
||||
|
||||
export async function getPersonalPublisherForUser(ctx: DbCtx, userId: Id<"users">) {
|
||||
try {
|
||||
return await ctx.db
|
||||
.query("publishers")
|
||||
@@ -102,10 +145,9 @@ export async function ensurePersonalPublisherForUser(
|
||||
const handle = derivePersonalPublisherHandle(user);
|
||||
let existing: Doc<"publishers"> | null = null;
|
||||
try {
|
||||
existing =
|
||||
user.personalPublisherId
|
||||
? await ctx.db.get(user.personalPublisherId)
|
||||
: await getPersonalPublisherForUser(ctx, user._id);
|
||||
existing = user.personalPublisherId
|
||||
? await ctx.db.get(user.personalPublisherId)
|
||||
: await getPersonalPublisherForUser(ctx, user._id);
|
||||
} catch (error) {
|
||||
if (!isMissingPublisherTableError(error)) throw error;
|
||||
return synthesizePersonalPublisher(user);
|
||||
@@ -193,7 +235,9 @@ export async function ensurePersonalPublisherForUser(
|
||||
|
||||
const existingMember = await ctx.db
|
||||
.query("publisherMembers")
|
||||
.withIndex("by_publisher_user", (q) => q.eq("publisherId", publisherId).eq("userId", user._id))
|
||||
.withIndex("by_publisher_user", (q) =>
|
||||
q.eq("publisherId", publisherId).eq("userId", user._id),
|
||||
)
|
||||
.unique();
|
||||
if (!existingMember) {
|
||||
await ctx.db.insert("publisherMembers", {
|
||||
@@ -290,15 +334,5 @@ export async function getOwnerPublisher(
|
||||
if (!params.ownerUserId) return null;
|
||||
const user = await ctx.db.get(params.ownerUserId);
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return null;
|
||||
if (user.personalPublisherId) {
|
||||
const publisher = await ctx.db.get(user.personalPublisherId);
|
||||
if (isPublisherActive(publisher)) return publisher;
|
||||
}
|
||||
try {
|
||||
const publisher = await getPersonalPublisherForUser(ctx, params.ownerUserId);
|
||||
if (isPublisherActive(publisher)) return publisher;
|
||||
} catch (error) {
|
||||
if (!isMissingPublisherTableError(error)) throw error;
|
||||
}
|
||||
return synthesizePersonalPublisher(user);
|
||||
return await getPersonalPublisherForUserOrFallback(ctx, user);
|
||||
}
|
||||
|
||||
@@ -47,4 +47,55 @@ describe("searchText", () => {
|
||||
it("normalize uses lowercase", () => {
|
||||
expect(__test.normalize("AbC")).toBe("abc");
|
||||
});
|
||||
|
||||
// CJK (Chinese, Japanese, Korean) support tests
|
||||
describe("CJK tokenization", () => {
|
||||
it("tokenizes Chinese text using Intl.Segmenter", () => {
|
||||
const tokens = tokenize("中文搜索");
|
||||
expect(tokens.length).toBeGreaterThan(0);
|
||||
expect(tokens).toContain("中文");
|
||||
expect(tokens).toContain("搜索");
|
||||
});
|
||||
|
||||
it("tokenizes mixed Chinese and English text", () => {
|
||||
const tokens = tokenize("React 组件开发");
|
||||
expect(tokens).toContain("react");
|
||||
expect(tokens.some((t) => t.includes("组") || t.includes("件"))).toBe(true);
|
||||
});
|
||||
|
||||
it("matches Chinese query tokens against Chinese skill names", () => {
|
||||
const queryTokens = tokenize("翻译");
|
||||
const skillName = "AI翻译助手";
|
||||
expect(matchesExactTokens(queryTokens, [skillName])).toBe(true);
|
||||
});
|
||||
|
||||
it("matches partial Chinese words", () => {
|
||||
const queryTokens = tokenize("助手");
|
||||
const skillName = "AI翻译助手";
|
||||
expect(matchesExactTokens(queryTokens, [skillName])).toBe(true);
|
||||
});
|
||||
|
||||
it("handles Japanese text", () => {
|
||||
const tokens = tokenize("こんにちは世界");
|
||||
expect(tokens.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("handles Korean text", () => {
|
||||
const tokens = tokenize("안녕하세요");
|
||||
expect(tokens.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("returns empty array for empty or whitespace-only input", () => {
|
||||
expect(tokenize("")).toEqual([]);
|
||||
expect(tokenize(" ")).toEqual([]);
|
||||
expect(tokenize("!!!")).toEqual([]);
|
||||
});
|
||||
|
||||
it("detects CJK language correctly", () => {
|
||||
expect(__test.detectCJKLanguage("中文")).toBe("zh");
|
||||
expect(__test.detectCJKLanguage("こんにちは")).toBe("ja");
|
||||
expect(__test.detectCJKLanguage("안녕하세요")).toBe("ko");
|
||||
expect(__test.detectCJKLanguage("hello")).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
+130
-3
@@ -1,12 +1,135 @@
|
||||
const WORD_RE = /[a-z0-9]+/g;
|
||||
const CJK_RE = /[\u4e00-\u9fff\u3400-\u4dbf\u3041-\u3096\u30a1-\u30fa\uac00-\ud7af]/;
|
||||
|
||||
const hasSegmenter = typeof Intl !== "undefined" && "Segmenter" in Intl;
|
||||
|
||||
let zhSegmenter: Intl.Segmenter | null = null;
|
||||
let jaSegmenter: Intl.Segmenter | null = null;
|
||||
let koSegmenter: Intl.Segmenter | null = null;
|
||||
|
||||
function getZhSegmenter(): Intl.Segmenter {
|
||||
if (!zhSegmenter) {
|
||||
zhSegmenter = new Intl.Segmenter("zh-CN", { granularity: "word" });
|
||||
}
|
||||
return zhSegmenter;
|
||||
}
|
||||
|
||||
function getJaSegmenter(): Intl.Segmenter {
|
||||
if (!jaSegmenter) {
|
||||
jaSegmenter = new Intl.Segmenter("ja", { granularity: "word" });
|
||||
}
|
||||
return jaSegmenter;
|
||||
}
|
||||
|
||||
function getKoSegmenter(): Intl.Segmenter {
|
||||
if (!koSegmenter) {
|
||||
koSegmenter = new Intl.Segmenter("ko", { granularity: "word" });
|
||||
}
|
||||
return koSegmenter;
|
||||
}
|
||||
|
||||
/**
|
||||
* Fallback: split CJK text into individual characters.
|
||||
* Used when Intl.Segmenter is unavailable (e.g. stripped V8 runtime).
|
||||
*/
|
||||
function segmentCJKByChar(text: string): string[] {
|
||||
const tokens: string[] = [];
|
||||
for (const ch of text) {
|
||||
if (CJK_RE.test(ch)) {
|
||||
tokens.push(ch);
|
||||
}
|
||||
}
|
||||
return tokens;
|
||||
}
|
||||
|
||||
function normalize(value: string) {
|
||||
return value.toLowerCase();
|
||||
}
|
||||
|
||||
/**
|
||||
* Detect the primary CJK language in a text
|
||||
* Returns 'zh' for Chinese, 'ja' for Japanese, 'ko' for Korean, or null
|
||||
*/
|
||||
function detectCJKLanguage(text: string): "zh" | "ja" | "ko" | null {
|
||||
const chineseCount = (text.match(/[\u4e00-\u9fff\u3400-\u4dbf]/g) || []).length;
|
||||
const hiraganaCount = (text.match(/[\u3040-\u309f]/g) || []).length;
|
||||
const katakanaCount = (text.match(/[\u30a0-\u30ff]/g) || []).length;
|
||||
const hangulCount = (text.match(/[\uac00-\ud7af]/g) || []).length;
|
||||
if (hiraganaCount + katakanaCount > 0) {
|
||||
return "ja";
|
||||
}
|
||||
if (hangulCount > 0) {
|
||||
return "ko";
|
||||
}
|
||||
if (chineseCount > 0) {
|
||||
return "zh";
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Segment CJK text using Intl.Segmenter, falling back to character-level
|
||||
* tokenization when the API is unavailable.
|
||||
*/
|
||||
function segmentCJK(text: string): string[] {
|
||||
if (!hasSegmenter) return segmentCJKByChar(text);
|
||||
|
||||
const lang = detectCJKLanguage(text);
|
||||
if (!lang) return [];
|
||||
|
||||
let segmenter: Intl.Segmenter;
|
||||
switch (lang) {
|
||||
case "ja":
|
||||
segmenter = getJaSegmenter();
|
||||
break;
|
||||
case "ko":
|
||||
segmenter = getKoSegmenter();
|
||||
break;
|
||||
default:
|
||||
segmenter = getZhSegmenter();
|
||||
}
|
||||
|
||||
const segments: string[] = [];
|
||||
for (const { segment, isWordLike } of segmenter.segment(text)) {
|
||||
const trimmed = segment.trim();
|
||||
if (trimmed && isWordLike) {
|
||||
segments.push(trimmed);
|
||||
}
|
||||
}
|
||||
return segments;
|
||||
}
|
||||
|
||||
/**
|
||||
* Tokenize text for search, supporting both English and CJK languages
|
||||
*
|
||||
* For English: uses word boundaries (whitespace, punctuation)
|
||||
* For CJK: uses Intl.Segmenter for proper word segmentation
|
||||
*/
|
||||
export function tokenize(value: string): string[] {
|
||||
if (!value) return [];
|
||||
return normalize(value).match(WORD_RE) ?? [];
|
||||
|
||||
const normalized = normalize(value);
|
||||
|
||||
if (!CJK_RE.test(normalized)) {
|
||||
return normalized.match(/[a-z0-9]+/g) ?? [];
|
||||
}
|
||||
|
||||
const tokens: string[] = [];
|
||||
|
||||
const parts = normalized.split(/([^\u4e00-\u9fff\u3400-\u4dbf\u3041-\u3096\u30a1-\u30fa\uac00-\ud7af]+)/g);
|
||||
|
||||
for (const part of parts) {
|
||||
if (!part.trim()) continue;
|
||||
|
||||
if (CJK_RE.test(part)) {
|
||||
const cjkTokens = segmentCJK(part);
|
||||
tokens.push(...cjkTokens);
|
||||
} else {
|
||||
const asciiTokens = part.match(/[a-z0-9]+/g) ?? [];
|
||||
tokens.push(...asciiTokens);
|
||||
}
|
||||
}
|
||||
|
||||
return tokens;
|
||||
}
|
||||
|
||||
export function matchesExactTokens(
|
||||
@@ -24,4 +147,8 @@ export function matchesExactTokens(
|
||||
);
|
||||
}
|
||||
|
||||
export const __test = { normalize, tokenize, matchesExactTokens };
|
||||
export const __test = {
|
||||
normalize,
|
||||
detectCJKLanguage,
|
||||
segmentCJKByChar,
|
||||
};
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { deriveSkillCapabilityTags } from "./skillCapabilityTags";
|
||||
|
||||
describe("deriveSkillCapabilityTags", () => {
|
||||
it("detects wallet, payment, and transaction authority from crypto skills", () => {
|
||||
const tags = deriveSkillCapabilityTags({
|
||||
slug: "paytoll",
|
||||
displayName: "PayToll",
|
||||
summary: "DeFi tools paid with x402 micro-payments.",
|
||||
frontmatter: {
|
||||
"requires.env": ["PRIVATE_KEY"],
|
||||
},
|
||||
readmeText:
|
||||
"Payment is the auth. Each tool call costs USDC. The wallet private key signs EIP-712 payment authorizations.",
|
||||
fileContents: [
|
||||
{
|
||||
path: "src/executor.ts",
|
||||
content:
|
||||
"walletClient.sendTransaction({}); if (result.type === 'approval_required') { log('Sending approval transaction...'); }",
|
||||
},
|
||||
],
|
||||
});
|
||||
|
||||
expect(tags).toEqual([
|
||||
"crypto",
|
||||
"requires-wallet",
|
||||
"can-make-purchases",
|
||||
"can-sign-transactions",
|
||||
"requires-sensitive-credentials",
|
||||
]);
|
||||
});
|
||||
|
||||
it("detects OAuth-backed external posting behavior", () => {
|
||||
const tags = deriveSkillCapabilityTags({
|
||||
slug: "social-poster",
|
||||
displayName: "Social Poster",
|
||||
frontmatter: {},
|
||||
readmeText:
|
||||
"Post a tweet for the user. Requires an OAuth 2.0 access token with tweet.write scope.",
|
||||
fileContents: [],
|
||||
});
|
||||
|
||||
expect(tags).toEqual([
|
||||
"requires-oauth-token",
|
||||
"requires-sensitive-credentials",
|
||||
"posts-externally",
|
||||
]);
|
||||
});
|
||||
|
||||
it("detects non-oauth API key skills that still need sensitive credentials", () => {
|
||||
const tags = deriveSkillCapabilityTags({
|
||||
slug: "minimax-usage",
|
||||
displayName: "Minimax Usage",
|
||||
frontmatter: {},
|
||||
readmeText:
|
||||
"Create a .env file with MINIMAX_CODING_API_KEY and MINIMAX_GROUP_ID, then send an authorization: Bearer header to the MiniMax endpoint.",
|
||||
fileContents: [],
|
||||
});
|
||||
|
||||
expect(tags).toEqual(["requires-sensitive-credentials"]);
|
||||
});
|
||||
|
||||
it("does not treat generic broadcast wording as a crypto transaction signal", () => {
|
||||
const tags = deriveSkillCapabilityTags({
|
||||
slug: "notify-bot",
|
||||
displayName: "Notify Bot",
|
||||
frontmatter: {},
|
||||
readmeText: "Broadcast notifications to Slack and email when incidents are opened.",
|
||||
fileContents: [],
|
||||
});
|
||||
|
||||
expect(tags).toEqual([]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,168 @@
|
||||
export const SKILL_CAPABILITY_TAGS = [
|
||||
"crypto",
|
||||
"requires-wallet",
|
||||
"can-make-purchases",
|
||||
"can-sign-transactions",
|
||||
"requires-oauth-token",
|
||||
"requires-sensitive-credentials",
|
||||
"posts-externally",
|
||||
] as const;
|
||||
|
||||
export type SkillCapabilityTag = (typeof SKILL_CAPABILITY_TAGS)[number];
|
||||
|
||||
function safeJson(value: unknown) {
|
||||
try {
|
||||
return JSON.stringify(value);
|
||||
} catch {
|
||||
return "";
|
||||
}
|
||||
}
|
||||
|
||||
function normalizeText(parts: Array<string | undefined>) {
|
||||
return parts
|
||||
.filter((part): part is string => typeof part === "string" && part.trim().length > 0)
|
||||
.join("\n")
|
||||
.toLowerCase();
|
||||
}
|
||||
|
||||
function matches(text: string, patterns: RegExp[]) {
|
||||
return patterns.some((pattern) => pattern.test(text));
|
||||
}
|
||||
|
||||
const CRYPTO_PATTERNS = [
|
||||
/\bcrypto\b/,
|
||||
/\bblockchain\b/,
|
||||
/\bdefi\b/,
|
||||
/\bon-?chain\b/,
|
||||
/\bwallet\b/,
|
||||
/\bprivate key\b/,
|
||||
/\berc20\b/,
|
||||
/\busdc\b/,
|
||||
/\beth(?:ereum)?\b/,
|
||||
/\bbase network\b/,
|
||||
/\barbitrum\b/,
|
||||
/\boptimism\b/,
|
||||
/\bpolygon\b/,
|
||||
/\bavalanche\b/,
|
||||
/\bsolana\b/,
|
||||
/\baave\b/,
|
||||
/\btoken balance\b/,
|
||||
/\bswap\b/,
|
||||
/\bbridge\b/,
|
||||
/\bliquidity\b/,
|
||||
/\bens\b/,
|
||||
/\bx402\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const WALLET_PATTERNS = [
|
||||
/\bprivate[_ -]?key\b/,
|
||||
/\bwallet\b/,
|
||||
/\bmnemonic\b/,
|
||||
/\bseed phrase\b/,
|
||||
/\bconfigured wallet\b/,
|
||||
/\bsigner\b/,
|
||||
/\beip-712\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const PURCHASE_PATTERNS = [
|
||||
/\bpay(?:ment|ments)?\b/,
|
||||
/\bpaid automatically\b/,
|
||||
/\bpay per call\b/,
|
||||
/\bmicro-?payments?\b/,
|
||||
/\bpayment required\b/,
|
||||
/\bcosts? \$\d/,
|
||||
/\bcharged?\b/,
|
||||
/\bpurchase\b/,
|
||||
/\bbuy(?:\s+(?:credits?|tokens?|coins?|nft|subscription|plan))\b/,
|
||||
/\bpayment checkout\b/,
|
||||
/\bone-?click checkout\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const TRANSACTION_PATTERNS = [
|
||||
/\bsign(?:ing)? (?:and )?(?:submit|send|broadcast)? ?transactions?\b/,
|
||||
/\bsendtransaction\b/,
|
||||
/\bapproval_required\b/,
|
||||
/\bon-?chain (?:tx|transaction)\b/,
|
||||
/\bexecute(?:s|d)? transaction\b/,
|
||||
/\bbroadcast (?:transaction|tx)\b/,
|
||||
/\btransaction broadcast\b/,
|
||||
/\bwalletclient\.sendtransaction\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const OAUTH_PATTERNS = [
|
||||
/\boauth(?: 2\.0)?\b/,
|
||||
/\baccess token\b/,
|
||||
/\brefresh token\b/,
|
||||
/\bbearer token\b/,
|
||||
/\btweet\.write\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const SENSITIVE_CREDENTIAL_PATTERNS = [
|
||||
/api[_ -]?key\b/,
|
||||
/\baccess token\b/,
|
||||
/\brefresh token\b/,
|
||||
/\bbearer token\b/,
|
||||
/\bsession (?:cookie|cookies)\b/,
|
||||
/\bauth(?:entication)? (?:cookie|cookies)\b/,
|
||||
/\bprivate[_ -]?key\b/,
|
||||
/\bmnemonic\b/,
|
||||
/\bseed phrase\b/,
|
||||
/\bsigner\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
const EXTERNAL_POST_PATTERNS = [
|
||||
/\bpost(?: a| this)? tweet\b/,
|
||||
/\breply to (?:this )?tweet\b/,
|
||||
/\bquote tweet\b/,
|
||||
/\bpost to (?:x|twitter)\b/,
|
||||
/\btwitter-post\b/,
|
||||
/\bpublish post\b/,
|
||||
] satisfies RegExp[];
|
||||
|
||||
export function deriveSkillCapabilityTags(params: {
|
||||
slug: string;
|
||||
displayName: string;
|
||||
summary?: string;
|
||||
frontmatter?: Record<string, unknown>;
|
||||
readmeText: string;
|
||||
fileContents?: Array<{ path: string; content: string }>;
|
||||
}): SkillCapabilityTag[] {
|
||||
const text = normalizeText([
|
||||
params.slug,
|
||||
params.displayName,
|
||||
params.summary,
|
||||
safeJson(params.frontmatter),
|
||||
params.readmeText,
|
||||
...(params.fileContents ?? []).map((file) => `${file.path}\n${file.content}`),
|
||||
]);
|
||||
|
||||
const tags = new Set<SkillCapabilityTag>();
|
||||
|
||||
const isCrypto = matches(text, CRYPTO_PATTERNS);
|
||||
const requiresWallet = matches(text, WALLET_PATTERNS);
|
||||
const canMakePurchases = matches(text, PURCHASE_PATTERNS);
|
||||
const canSignTransactions = matches(text, TRANSACTION_PATTERNS);
|
||||
const requiresOauthToken = matches(text, OAUTH_PATTERNS);
|
||||
const requiresSensitiveCredentials = matches(text, SENSITIVE_CREDENTIAL_PATTERNS);
|
||||
const postsExternally = matches(text, EXTERNAL_POST_PATTERNS);
|
||||
|
||||
if (isCrypto) tags.add("crypto");
|
||||
if (requiresWallet) tags.add("requires-wallet");
|
||||
if (canMakePurchases) tags.add("can-make-purchases");
|
||||
if (canSignTransactions) tags.add("can-sign-transactions");
|
||||
if (requiresOauthToken) tags.add("requires-oauth-token");
|
||||
if (requiresSensitiveCredentials) tags.add("requires-sensitive-credentials");
|
||||
if (postsExternally) tags.add("posts-externally");
|
||||
|
||||
if (canSignTransactions || canMakePurchases) {
|
||||
tags.add("crypto");
|
||||
}
|
||||
if (canSignTransactions) {
|
||||
tags.add("requires-wallet");
|
||||
}
|
||||
if (requiresWallet || canSignTransactions || requiresOauthToken) {
|
||||
tags.add("requires-sensitive-credentials");
|
||||
}
|
||||
|
||||
return SKILL_CAPABILITY_TAGS.filter((tag) => tags.has(tag));
|
||||
}
|
||||
@@ -1,4 +1,5 @@
|
||||
import { ConvexError } from "convex/values";
|
||||
import { normalizeTextContentType } from "clawhub-schema";
|
||||
import semver from "semver";
|
||||
import { api, internal } from "../_generated/api";
|
||||
import type { Doc, Id } from "../_generated/dataModel";
|
||||
@@ -8,6 +9,13 @@ import { generateChangelogForPublish } from "./changelog";
|
||||
import { generateEmbedding } from "./embeddings";
|
||||
import { requireGitHubAccountAge } from "./githubAccount";
|
||||
import type { PublicUser } from "./public";
|
||||
import {
|
||||
findOversizedPublishFile,
|
||||
getPublishFileSizeError,
|
||||
getPublishTotalSizeError,
|
||||
MAX_PUBLISH_TOTAL_BYTES,
|
||||
} from "./publishLimits";
|
||||
import { deriveSkillCapabilityTags } from "./skillCapabilityTags";
|
||||
import {
|
||||
computeQualitySignals,
|
||||
evaluateQuality,
|
||||
@@ -29,12 +37,6 @@ import {
|
||||
import { generateSkillSummary } from "./skillSummary";
|
||||
import { runStaticPublishScan } from "./staticPublishScan";
|
||||
import type { WebhookSkillPayload } from "./webhooks";
|
||||
import {
|
||||
findOversizedPublishFile,
|
||||
getPublishFileSizeError,
|
||||
getPublishTotalSizeError,
|
||||
MAX_PUBLISH_TOTAL_BYTES,
|
||||
} from "./publishLimits";
|
||||
|
||||
const MAX_FILES_FOR_EMBEDDING = 40;
|
||||
const QUALITY_WINDOW_MS = 24 * 60 * 60 * 1000;
|
||||
@@ -112,6 +114,7 @@ export async function publishVersionForUser(
|
||||
const sanitizedFiles = args.files.map((file) => ({
|
||||
...file,
|
||||
path: sanitizePath(file.path),
|
||||
contentType: normalizeTextContentType(file.path, file.contentType),
|
||||
}));
|
||||
if (sanitizedFiles.some((file) => !file.path)) {
|
||||
throw new ConvexError("Invalid file paths");
|
||||
@@ -246,6 +249,14 @@ export async function publishVersionForUser(
|
||||
readme: readmeText,
|
||||
otherFiles,
|
||||
});
|
||||
const capabilityTags = deriveSkillCapabilityTags({
|
||||
slug,
|
||||
displayName,
|
||||
summary,
|
||||
frontmatter,
|
||||
readmeText,
|
||||
fileContents,
|
||||
});
|
||||
|
||||
const fingerprintPromise = hashSkillFiles(
|
||||
publishFiles.map((file) => ({ path: file.path, sha256: file.sha256 })),
|
||||
@@ -298,6 +309,7 @@ export async function publishVersionForUser(
|
||||
clawdis,
|
||||
license: PLATFORM_SKILL_LICENSE,
|
||||
},
|
||||
capabilityTags,
|
||||
summary,
|
||||
staticScan,
|
||||
embedding,
|
||||
|
||||
@@ -6,6 +6,8 @@ function pick<T extends Record<string, unknown>, K extends keyof T>(obj: T, keys
|
||||
return Object.fromEntries(keys.map((k) => [k, obj[k]])) as Pick<T, K>;
|
||||
}
|
||||
|
||||
type SharedSkillKey = Extract<keyof Doc<"skills">, keyof Doc<"skillSearchDigest">>;
|
||||
|
||||
/**
|
||||
* Fields shared 1:1 between `skills` and `skillSearchDigest` (same name,
|
||||
* same type). Used by both `extractDigestFields` and `digestToHydratableSkill`
|
||||
@@ -22,6 +24,7 @@ const SHARED_KEYS = [
|
||||
"latestVersionId",
|
||||
"latestVersionSummary",
|
||||
"tags",
|
||||
"capabilityTags",
|
||||
"badges",
|
||||
"stats",
|
||||
"statsDownloads",
|
||||
@@ -34,7 +37,7 @@ const SHARED_KEYS = [
|
||||
"moderationReason",
|
||||
"createdAt",
|
||||
"updatedAt",
|
||||
] as const satisfies readonly (keyof Doc<"skills"> & keyof Doc<"skillSearchDigest">)[];
|
||||
] as const satisfies readonly SharedSkillKey[];
|
||||
|
||||
/** Fields stored in the skillSearchDigest table. */
|
||||
export type SkillSearchDigestFields = Pick<Doc<"skills">, (typeof SHARED_KEYS)[number]> & {
|
||||
@@ -121,8 +124,7 @@ export function digestToOwnerInfo(
|
||||
// Empty string means backfilled but owner has no handle.
|
||||
// Use userId as fallback handle, matching the live getOwnerInfo path.
|
||||
const handle = digest.ownerHandle || undefined;
|
||||
const fallbackHandle =
|
||||
handle ?? String(digest.ownerPublisherId ?? digest.ownerUserId);
|
||||
const fallbackHandle = handle ?? String(digest.ownerPublisherId ?? digest.ownerUserId);
|
||||
const resolvedHandle = handle ?? fallbackHandle;
|
||||
// Determine if we have real profile data (deactivated/deleted owners have
|
||||
// all profile fields undefined, while handle-less visible owners still have
|
||||
|
||||
+27
-11
@@ -10,18 +10,34 @@ type SkillStatDeltas = {
|
||||
installsAllTime?: number;
|
||||
};
|
||||
|
||||
/**
|
||||
* Read the canonical value of a migrated stat field from a skill document.
|
||||
*
|
||||
* Top-level fields (`statsDownloads`, etc.) are the source of truth — they are
|
||||
* indexable and kept up-to-date by the event pipeline. The nested `stats.*`
|
||||
* fields are only used as a fallback for pre-migration documents where the
|
||||
* top-level field is still `undefined`.
|
||||
*
|
||||
* All code that reads a migrated stat value should go through this function
|
||||
* rather than accessing `skill.stats.*` directly.
|
||||
*/
|
||||
export function readCanonicalStat(
|
||||
skill: Doc<"skills">,
|
||||
field: "downloads" | "stars" | "installsCurrent" | "installsAllTime",
|
||||
): number {
|
||||
const topLevelKey = `stats${field[0].toUpperCase()}${field.slice(1)}` as
|
||||
| "statsDownloads"
|
||||
| "statsStars"
|
||||
| "statsInstallsCurrent"
|
||||
| "statsInstallsAllTime";
|
||||
return typeof skill[topLevelKey] === "number" ? skill[topLevelKey]! : (skill.stats[field] ?? 0);
|
||||
}
|
||||
|
||||
export function applySkillStatDeltas(skill: Doc<"skills">, deltas: SkillStatDeltas) {
|
||||
const currentDownloads =
|
||||
typeof skill.statsDownloads === "number" ? skill.statsDownloads : skill.stats.downloads;
|
||||
const currentStars = typeof skill.statsStars === "number" ? skill.statsStars : skill.stats.stars;
|
||||
const currentInstallsCurrent =
|
||||
typeof skill.statsInstallsCurrent === "number"
|
||||
? skill.statsInstallsCurrent
|
||||
: (skill.stats.installsCurrent ?? 0);
|
||||
const currentInstallsAllTime =
|
||||
typeof skill.statsInstallsAllTime === "number"
|
||||
? skill.statsInstallsAllTime
|
||||
: (skill.stats.installsAllTime ?? 0);
|
||||
const currentDownloads = readCanonicalStat(skill, "downloads");
|
||||
const currentStars = readCanonicalStat(skill, "stars");
|
||||
const currentInstallsCurrent = readCanonicalStat(skill, "installsCurrent");
|
||||
const currentInstallsAllTime = readCanonicalStat(skill, "installsAllTime");
|
||||
|
||||
const currentComments = skill.stats.comments;
|
||||
const nextDownloads = Math.max(0, currentDownloads + (deltas.downloads ?? 0));
|
||||
|
||||
@@ -150,7 +150,10 @@ describe("skillZip", () => {
|
||||
]);
|
||||
const unzipped = unzipSync(zip);
|
||||
|
||||
expect(Object.keys(unzipped).sort()).toEqual(["package/dist/index.js", "package/package.json"]);
|
||||
expect(Object.keys(unzipped).sort()).toEqual([
|
||||
"package/dist/index.js",
|
||||
"package/package.json",
|
||||
]);
|
||||
expect(unzipped["_meta.json"]).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { ConvexError } from "convex/values";
|
||||
import { normalizeTextContentType } from "clawhub-schema";
|
||||
import semver from "semver";
|
||||
import { internal } from "../_generated/api";
|
||||
import type { Doc, Id } from "../_generated/dataModel";
|
||||
@@ -101,7 +102,11 @@ export async function publishSoulVersionForUser(
|
||||
const sanitizedFiles = args.files.map((file) => {
|
||||
const path = sanitizePath(file.path);
|
||||
if (!path) throw new ConvexError("Invalid file paths");
|
||||
return { ...file, path };
|
||||
return {
|
||||
...file,
|
||||
path,
|
||||
contentType: normalizeTextContentType(file.path, file.contentType),
|
||||
};
|
||||
});
|
||||
const publishFiles = sanitizedFiles.filter((file) => !isMacJunkPath(file.path));
|
||||
if (publishFiles.some((file) => !isTextFile(file.path, file.contentType ?? undefined))) {
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
import type { Doc, Id } from "../_generated/dataModel";
|
||||
import type { MutationCtx } from "../_generated/server";
|
||||
|
||||
function getSkillContribution(skill: Doc<"skills">) {
|
||||
if (skill.softDeletedAt) {
|
||||
return { publishedSkills: 0, totalStars: 0, totalDownloads: 0 };
|
||||
}
|
||||
|
||||
return {
|
||||
publishedSkills: 1,
|
||||
totalStars: skill.stats?.stars ?? 0,
|
||||
totalDownloads: skill.stats?.downloads ?? 0,
|
||||
};
|
||||
}
|
||||
|
||||
async function patchUserStats(
|
||||
ctx: Pick<MutationCtx, "db">,
|
||||
userId: Id<"users">,
|
||||
delta: { publishedSkills: number; totalStars: number; totalDownloads: number },
|
||||
) {
|
||||
const user = await ctx.db.get(userId);
|
||||
if (!user) return;
|
||||
|
||||
await ctx.db.patch(userId, {
|
||||
publishedSkills: Math.max(0, (user.publishedSkills ?? 0) + delta.publishedSkills),
|
||||
totalStars: Math.max(0, (user.totalStars ?? 0) + delta.totalStars),
|
||||
totalDownloads: Math.max(0, (user.totalDownloads ?? 0) + delta.totalDownloads),
|
||||
});
|
||||
}
|
||||
|
||||
export async function adjustUserSkillStatsForSkillChange(
|
||||
ctx: Pick<MutationCtx, "db">,
|
||||
previousSkill: Doc<"skills"> | null | undefined,
|
||||
nextSkill: Doc<"skills"> | null | undefined,
|
||||
) {
|
||||
if (!previousSkill && !nextSkill) return;
|
||||
|
||||
const prevOwnerId = previousSkill?.ownerUserId ?? null;
|
||||
const nextOwnerId = nextSkill?.ownerUserId ?? null;
|
||||
const prevContribution = previousSkill ? getSkillContribution(previousSkill) : null;
|
||||
const nextContribution = nextSkill ? getSkillContribution(nextSkill) : null;
|
||||
|
||||
if (prevOwnerId && prevOwnerId === nextOwnerId) {
|
||||
await patchUserStats(ctx, prevOwnerId, {
|
||||
publishedSkills: (nextContribution?.publishedSkills ?? 0) - (prevContribution?.publishedSkills ?? 0),
|
||||
totalStars: (nextContribution?.totalStars ?? 0) - (prevContribution?.totalStars ?? 0),
|
||||
totalDownloads: (nextContribution?.totalDownloads ?? 0) - (prevContribution?.totalDownloads ?? 0),
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
if (prevOwnerId) {
|
||||
await patchUserStats(ctx, prevOwnerId, {
|
||||
publishedSkills: -(prevContribution?.publishedSkills ?? 0),
|
||||
totalStars: -(prevContribution?.totalStars ?? 0),
|
||||
totalDownloads: -(prevContribution?.totalDownloads ?? 0),
|
||||
});
|
||||
}
|
||||
|
||||
if (nextOwnerId) {
|
||||
await patchUserStats(ctx, nextOwnerId, {
|
||||
publishedSkills: nextContribution?.publishedSkills ?? 0,
|
||||
totalStars: nextContribution?.totalStars ?? 0,
|
||||
totalDownloads: nextContribution?.totalDownloads ?? 0,
|
||||
});
|
||||
}
|
||||
}
|
||||
+9
-3
@@ -323,7 +323,10 @@ export const evaluatePackageReleaseWithLlm = internalAction({
|
||||
const content = await blob.text();
|
||||
fileContents.push({ path: f.path, content });
|
||||
const lower = f.path.toLowerCase();
|
||||
if (!readmeContent && (lower === "readme.md" || lower === "readme.mdx" || lower === "readme.markdown")) {
|
||||
if (
|
||||
!readmeContent &&
|
||||
(lower === "readme.md" || lower === "readme.mdx" || lower === "readme.markdown")
|
||||
) {
|
||||
readmeContent = content;
|
||||
}
|
||||
} catch {
|
||||
@@ -332,8 +335,11 @@ export const evaluatePackageReleaseWithLlm = internalAction({
|
||||
}
|
||||
|
||||
if (!readmeContent) {
|
||||
const packageJsonText = fileContents.find((entry) => entry.path.toLowerCase() === "package.json")?.content;
|
||||
readmeContent = packageJsonText ?? `# ${pkg.displayName}\n\n${release.summary ?? pkg.summary ?? pkg.name}`;
|
||||
const packageJsonText = fileContents.find(
|
||||
(entry) => entry.path.toLowerCase() === "package.json",
|
||||
)?.content;
|
||||
readmeContent =
|
||||
packageJsonText ?? `# ${pkg.displayName}\n\n${release.summary ?? pkg.summary ?? pkg.name}`;
|
||||
}
|
||||
|
||||
const allContent = [readmeContent, ...fileContents.map((f) => f.content)].join("\n");
|
||||
|
||||
@@ -7,6 +7,10 @@ vi.mock("./_generated/api", () => ({
|
||||
getSkillBackfillPageInternal: Symbol("getSkillBackfillPageInternal"),
|
||||
applySkillBackfillPatchInternal: Symbol("applySkillBackfillPatchInternal"),
|
||||
backfillSkillSummariesInternal: Symbol("backfillSkillSummariesInternal"),
|
||||
getUserStatsBackfillPageInternal: Symbol("getUserStatsBackfillPageInternal"),
|
||||
getUserOwnedSkillsBackfillPageInternal: Symbol("getUserOwnedSkillsBackfillPageInternal"),
|
||||
applyUserStatsBackfillPatchInternal: Symbol("applyUserStatsBackfillPatchInternal"),
|
||||
backfillUserStatsInternal: Symbol("backfillUserStatsInternal"),
|
||||
getSkillFingerprintBackfillPageInternal: Symbol("getSkillFingerprintBackfillPageInternal"),
|
||||
applySkillFingerprintBackfillPatchInternal: Symbol(
|
||||
"applySkillFingerprintBackfillPatchInternal",
|
||||
@@ -36,6 +40,7 @@ const {
|
||||
backfillLatestVersionSummaryInternal,
|
||||
backfillSkillFingerprintsInternalHandler,
|
||||
backfillSkillSummariesInternalHandler,
|
||||
backfillUserStatsInternalHandler,
|
||||
cleanupEmptySkillsInternalHandler,
|
||||
nominateEmptySkillSpammersInternalHandler,
|
||||
upsertSkillBadgeRecordInternal,
|
||||
@@ -259,6 +264,63 @@ describe("maintenance backfill", () => {
|
||||
});
|
||||
expect(runAfter).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("backfills denormalized user hover stats from indexed owner pages", async () => {
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
items: [{ _id: "users:1" }],
|
||||
cursor: null,
|
||||
isDone: true,
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
items: [
|
||||
{ stats: { stars: 4, downloads: 30 }, softDeletedAt: undefined },
|
||||
{ stats: { stars: 2, downloads: 10 }, softDeletedAt: 123 },
|
||||
{ stats: { stars: 1, downloads: 5 }, softDeletedAt: undefined },
|
||||
],
|
||||
cursor: null,
|
||||
isDone: true,
|
||||
});
|
||||
const runMutation = vi.fn().mockResolvedValue({ ok: true });
|
||||
|
||||
const result = await backfillUserStatsInternalHandler(
|
||||
{ runQuery, runMutation } as never,
|
||||
{ batchSize: 10, skillBatchSize: 50, maxBatches: 1 },
|
||||
);
|
||||
|
||||
expect(result).toEqual({
|
||||
ok: true,
|
||||
stats: {
|
||||
usersScanned: 1,
|
||||
usersPatched: 1,
|
||||
},
|
||||
isDone: true,
|
||||
cursor: null,
|
||||
});
|
||||
expect(runQuery).toHaveBeenNthCalledWith(1, internal.maintenance.getUserStatsBackfillPageInternal, {
|
||||
cursor: undefined,
|
||||
batchSize: 10,
|
||||
});
|
||||
expect(runQuery).toHaveBeenNthCalledWith(
|
||||
2,
|
||||
internal.maintenance.getUserOwnedSkillsBackfillPageInternal,
|
||||
{
|
||||
ownerUserId: "users:1",
|
||||
cursor: undefined,
|
||||
batchSize: 50,
|
||||
},
|
||||
);
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
internal.maintenance.applyUserStatsBackfillPatchInternal,
|
||||
{
|
||||
userId: "users:1",
|
||||
publishedSkills: 2,
|
||||
totalStars: 5,
|
||||
totalDownloads: 35,
|
||||
},
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("maintenance badge denormalization", () => {
|
||||
|
||||
+375
-1
@@ -5,13 +5,14 @@ import type { ActionCtx } from "./_generated/server";
|
||||
import { action, internalAction, internalMutation, internalQuery } from "./functions";
|
||||
import { assertRole, requireUserFromAction } from "./lib/access";
|
||||
import { buildSkillSummaryBackfillPatch, type ParsedSkillData } from "./lib/skillBackfill";
|
||||
import { deriveSkillCapabilityTags } from "./lib/skillCapabilityTags";
|
||||
import {
|
||||
computeQualitySignals,
|
||||
evaluateQuality,
|
||||
getTrustTier,
|
||||
type TrustTier,
|
||||
} from "./lib/skillQuality";
|
||||
import { hashSkillFiles } from "./lib/skills";
|
||||
import { hashSkillFiles, isTextFile } from "./lib/skills";
|
||||
import { computeIsSuspicious } from "./lib/skillSafety";
|
||||
import { extractDigestFields } from "./lib/skillSearchDigest";
|
||||
import { generateSkillSummary } from "./lib/skillSummary";
|
||||
@@ -22,6 +23,7 @@ const DEFAULT_MAX_BATCHES = 20;
|
||||
const MAX_MAX_BATCHES = 200;
|
||||
const DEFAULT_EMPTY_SKILL_MAX_README_BYTES = 8000;
|
||||
const DEFAULT_EMPTY_SKILL_NOMINATION_THRESHOLD = 3;
|
||||
const DEFAULT_CAPABILITY_BACKFILL_DELAY_MS = 500;
|
||||
const PLATFORM_SKILL_LICENSE = "MIT-0" as const;
|
||||
|
||||
type BackfillStats = {
|
||||
@@ -34,6 +36,11 @@ type BackfillStats = {
|
||||
missingStorageBlob: number;
|
||||
};
|
||||
|
||||
type UserStatsBackfillStats = {
|
||||
usersScanned: number;
|
||||
usersPatched: number;
|
||||
};
|
||||
|
||||
type BackfillPageItem =
|
||||
| {
|
||||
kind: "ok";
|
||||
@@ -55,6 +62,18 @@ type BackfillPageResult = {
|
||||
isDone: boolean;
|
||||
};
|
||||
|
||||
type UserStatsBackfillPageResult = {
|
||||
items: Array<Pick<Doc<"users">, "_id">>;
|
||||
cursor: string | null;
|
||||
isDone: boolean;
|
||||
};
|
||||
|
||||
type UserOwnedSkillsBackfillPageResult = {
|
||||
items: Array<Pick<Doc<"skills">, "stats" | "softDeletedAt">>;
|
||||
cursor: string | null;
|
||||
isDone: boolean;
|
||||
};
|
||||
|
||||
export const getSkillBackfillPageInternal = internalQuery({
|
||||
args: {
|
||||
cursor: v.optional(v.string()),
|
||||
@@ -134,6 +153,65 @@ export const applySkillBackfillPatchInternal = internalMutation({
|
||||
},
|
||||
});
|
||||
|
||||
export const getUserStatsBackfillPageInternal = internalQuery({
|
||||
args: {
|
||||
cursor: v.optional(v.string()),
|
||||
batchSize: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<UserStatsBackfillPageResult> => {
|
||||
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
|
||||
const { page, isDone, continueCursor } = await ctx.db
|
||||
.query("users")
|
||||
.order("asc")
|
||||
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
|
||||
return {
|
||||
items: page.map((user) => ({ _id: user._id })),
|
||||
cursor: continueCursor,
|
||||
isDone,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const getUserOwnedSkillsBackfillPageInternal = internalQuery({
|
||||
args: {
|
||||
ownerUserId: v.id("users"),
|
||||
cursor: v.optional(v.string()),
|
||||
batchSize: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<UserOwnedSkillsBackfillPageResult> => {
|
||||
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
|
||||
const { page, isDone, continueCursor } = await ctx.db
|
||||
.query("skills")
|
||||
.withIndex("by_owner", (q) => q.eq("ownerUserId", args.ownerUserId))
|
||||
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
|
||||
return {
|
||||
items: page.map((skill) => ({
|
||||
stats: skill.stats,
|
||||
softDeletedAt: skill.softDeletedAt,
|
||||
})),
|
||||
cursor: continueCursor,
|
||||
isDone,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const applyUserStatsBackfillPatchInternal = internalMutation({
|
||||
args: {
|
||||
userId: v.id("users"),
|
||||
publishedSkills: v.number(),
|
||||
totalStars: v.number(),
|
||||
totalDownloads: v.number(),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
await ctx.db.patch(args.userId, {
|
||||
publishedSkills: args.publishedSkills,
|
||||
totalStars: args.totalStars,
|
||||
totalDownloads: args.totalDownloads,
|
||||
});
|
||||
return { ok: true as const };
|
||||
},
|
||||
});
|
||||
|
||||
export type BackfillActionArgs = {
|
||||
dryRun?: boolean;
|
||||
batchSize?: number;
|
||||
@@ -149,6 +227,20 @@ export type BackfillActionResult = {
|
||||
cursor: string | null;
|
||||
};
|
||||
|
||||
export type UserStatsBackfillActionArgs = {
|
||||
batchSize?: number;
|
||||
skillBatchSize?: number;
|
||||
maxBatches?: number;
|
||||
cursor?: string;
|
||||
};
|
||||
|
||||
export type UserStatsBackfillActionResult = {
|
||||
ok: true;
|
||||
stats: UserStatsBackfillStats;
|
||||
isDone: boolean;
|
||||
cursor: string | null;
|
||||
};
|
||||
|
||||
export async function backfillSkillSummariesInternalHandler(
|
||||
ctx: ActionCtx,
|
||||
args: BackfillActionArgs,
|
||||
@@ -244,6 +336,73 @@ export async function backfillSkillSummariesInternalHandler(
|
||||
return { ok: true as const, stats: totals, isDone, cursor };
|
||||
}
|
||||
|
||||
export async function backfillUserStatsInternalHandler(
|
||||
ctx: ActionCtx,
|
||||
args: UserStatsBackfillActionArgs,
|
||||
): Promise<UserStatsBackfillActionResult> {
|
||||
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
|
||||
const skillBatchSize = clampInt(args.skillBatchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
|
||||
const maxBatches = clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES);
|
||||
const totals: UserStatsBackfillStats = {
|
||||
usersScanned: 0,
|
||||
usersPatched: 0,
|
||||
};
|
||||
|
||||
let cursor: string | null = args.cursor ?? null;
|
||||
let isDone = false;
|
||||
|
||||
for (let i = 0; i < maxBatches; i++) {
|
||||
const page = (await ctx.runQuery(internal.maintenance.getUserStatsBackfillPageInternal, {
|
||||
cursor: cursor ?? undefined,
|
||||
batchSize,
|
||||
})) as UserStatsBackfillPageResult;
|
||||
|
||||
cursor = page.cursor;
|
||||
isDone = page.isDone;
|
||||
|
||||
for (const user of page.items) {
|
||||
totals.usersScanned++;
|
||||
let ownedSkillsCursor: string | null = null;
|
||||
let userPublishedSkills = 0;
|
||||
let userTotalStars = 0;
|
||||
let userTotalDownloads = 0;
|
||||
|
||||
while (true) {
|
||||
const skillPage = (await ctx.runQuery(
|
||||
internal.maintenance.getUserOwnedSkillsBackfillPageInternal,
|
||||
{
|
||||
ownerUserId: user._id,
|
||||
cursor: ownedSkillsCursor ?? undefined,
|
||||
batchSize: skillBatchSize,
|
||||
},
|
||||
)) as UserOwnedSkillsBackfillPageResult;
|
||||
|
||||
for (const skill of skillPage.items) {
|
||||
if (skill.softDeletedAt) continue;
|
||||
userPublishedSkills += 1;
|
||||
userTotalStars += skill.stats?.stars ?? 0;
|
||||
userTotalDownloads += skill.stats?.downloads ?? 0;
|
||||
}
|
||||
|
||||
if (skillPage.isDone) break;
|
||||
ownedSkillsCursor = skillPage.cursor;
|
||||
}
|
||||
|
||||
await ctx.runMutation(internal.maintenance.applyUserStatsBackfillPatchInternal, {
|
||||
userId: user._id,
|
||||
publishedSkills: userPublishedSkills,
|
||||
totalStars: userTotalStars,
|
||||
totalDownloads: userTotalDownloads,
|
||||
});
|
||||
totals.usersPatched++;
|
||||
}
|
||||
|
||||
if (isDone) break;
|
||||
}
|
||||
|
||||
return { ok: true as const, stats: totals, isDone, cursor };
|
||||
}
|
||||
|
||||
export const backfillSkillSummariesInternal = internalAction({
|
||||
args: {
|
||||
dryRun: v.optional(v.boolean()),
|
||||
@@ -255,6 +414,16 @@ export const backfillSkillSummariesInternal = internalAction({
|
||||
handler: backfillSkillSummariesInternalHandler,
|
||||
});
|
||||
|
||||
export const backfillUserStatsInternal = internalAction({
|
||||
args: {
|
||||
batchSize: v.optional(v.number()),
|
||||
skillBatchSize: v.optional(v.number()),
|
||||
maxBatches: v.optional(v.number()),
|
||||
cursor: v.optional(v.string()),
|
||||
},
|
||||
handler: backfillUserStatsInternalHandler,
|
||||
});
|
||||
|
||||
export const backfillSkillSummaries: ReturnType<typeof action> = action({
|
||||
args: {
|
||||
dryRun: v.optional(v.boolean()),
|
||||
@@ -319,6 +488,211 @@ export const continueSkillSummaryBackfillJobInternal = internalAction({
|
||||
},
|
||||
});
|
||||
|
||||
type CapabilityBackfillStats = {
|
||||
skillsScanned: number;
|
||||
skillsPatched: number;
|
||||
versionsPatched: number;
|
||||
missingVersions: number;
|
||||
missingStorageBlob: number;
|
||||
};
|
||||
|
||||
type CapabilityBackfillResult = {
|
||||
ok: true;
|
||||
stats: CapabilityBackfillStats;
|
||||
cursor: string | null;
|
||||
isDone: boolean;
|
||||
};
|
||||
|
||||
export const applySkillCapabilityTagsInternal = internalMutation({
|
||||
args: {
|
||||
skillId: v.id("skills"),
|
||||
versionId: v.id("skillVersions"),
|
||||
capabilityTags: v.array(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const version = await ctx.db.get(args.versionId);
|
||||
if (!version) return { ok: false as const, reason: "missing_version" as const };
|
||||
const skill = await ctx.db.get(args.skillId);
|
||||
if (!skill) return { ok: false as const, reason: "missing_skill" as const };
|
||||
|
||||
const normalizedTags = [...new Set(args.capabilityTags)];
|
||||
let versionPatched = false;
|
||||
let skillPatched = false;
|
||||
|
||||
if (JSON.stringify(version.capabilityTags ?? []) !== JSON.stringify(normalizedTags)) {
|
||||
await ctx.db.patch(version._id, {
|
||||
capabilityTags: normalizedTags.length ? normalizedTags : undefined,
|
||||
});
|
||||
versionPatched = true;
|
||||
}
|
||||
|
||||
if (
|
||||
skill.latestVersionId === version._id &&
|
||||
JSON.stringify(skill.capabilityTags ?? []) !== JSON.stringify(normalizedTags)
|
||||
) {
|
||||
await ctx.db.patch(skill._id, {
|
||||
capabilityTags: normalizedTags.length ? normalizedTags : undefined,
|
||||
updatedAt: Date.now(),
|
||||
});
|
||||
skillPatched = true;
|
||||
}
|
||||
|
||||
return { ok: true as const, versionPatched, skillPatched };
|
||||
},
|
||||
});
|
||||
|
||||
export async function backfillSkillCapabilityTagsInternalHandler(
|
||||
ctx: ActionCtx,
|
||||
args: {
|
||||
dryRun?: boolean;
|
||||
cursor?: string;
|
||||
batchSize?: number;
|
||||
maxBatches?: number;
|
||||
delayMs?: number;
|
||||
},
|
||||
): Promise<CapabilityBackfillResult> {
|
||||
const dryRun = Boolean(args.dryRun);
|
||||
const batchSize = clampInt(args.batchSize ?? DEFAULT_BATCH_SIZE, 1, MAX_BATCH_SIZE);
|
||||
const maxBatches = dryRun
|
||||
? clampInt(args.maxBatches ?? DEFAULT_MAX_BATCHES, 1, MAX_MAX_BATCHES)
|
||||
: 1;
|
||||
|
||||
const stats: CapabilityBackfillStats = {
|
||||
skillsScanned: 0,
|
||||
skillsPatched: 0,
|
||||
versionsPatched: 0,
|
||||
missingVersions: 0,
|
||||
missingStorageBlob: 0,
|
||||
};
|
||||
|
||||
let cursor = args.cursor ?? null;
|
||||
let isDone = false;
|
||||
|
||||
for (let batchIndex = 0; batchIndex < maxBatches; batchIndex += 1) {
|
||||
const page = await ctx.runQuery(internal.maintenance.getSkillBackfillPageInternal, {
|
||||
cursor: cursor ?? undefined,
|
||||
batchSize,
|
||||
});
|
||||
|
||||
cursor = page.cursor;
|
||||
isDone = page.isDone;
|
||||
|
||||
for (const item of page.items) {
|
||||
if (item.kind !== "ok") {
|
||||
if (item.kind === "missingVersionDoc" || item.kind === "missingLatestVersion") {
|
||||
stats.missingVersions += 1;
|
||||
}
|
||||
continue;
|
||||
}
|
||||
|
||||
stats.skillsScanned += 1;
|
||||
|
||||
const version = (await ctx.runQuery(internal.skills.getVersionByIdInternal, {
|
||||
versionId: item.versionId,
|
||||
})) as Doc<"skillVersions"> | null;
|
||||
if (!version) {
|
||||
stats.missingVersions += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const readmeBlob = await ctx.storage.get(item.readmeStorageId);
|
||||
if (!readmeBlob) {
|
||||
stats.missingStorageBlob += 1;
|
||||
continue;
|
||||
}
|
||||
|
||||
const readmeText = await readmeBlob.text();
|
||||
const fileContents: Array<{ path: string; content: string }> = [];
|
||||
let hasMissingTextBlob = false;
|
||||
for (const file of version.files) {
|
||||
const lower = file.path.toLowerCase();
|
||||
if (lower === "skill.md" || lower === "skills.md") continue;
|
||||
if (!isTextFile(file.path, file.contentType ?? undefined)) continue;
|
||||
const blob = await ctx.storage.get(file.storageId);
|
||||
if (!blob) {
|
||||
stats.missingStorageBlob += 1;
|
||||
hasMissingTextBlob = true;
|
||||
break;
|
||||
}
|
||||
fileContents.push({ path: file.path, content: await blob.text() });
|
||||
}
|
||||
|
||||
if (hasMissingTextBlob) continue;
|
||||
|
||||
const capabilityTags = deriveSkillCapabilityTags({
|
||||
slug: item.skillSlug,
|
||||
displayName: item.skillDisplayName,
|
||||
summary: item.skillSummary ?? undefined,
|
||||
frontmatter: item.versionParsed?.frontmatter,
|
||||
readmeText,
|
||||
fileContents,
|
||||
});
|
||||
|
||||
if (dryRun) continue;
|
||||
|
||||
const result = await ctx.runMutation(internal.maintenance.applySkillCapabilityTagsInternal, {
|
||||
skillId: item.skillId,
|
||||
versionId: item.versionId,
|
||||
capabilityTags,
|
||||
});
|
||||
|
||||
if (result.ok) {
|
||||
if (result.skillPatched) stats.skillsPatched += 1;
|
||||
if (result.versionPatched) stats.versionsPatched += 1;
|
||||
}
|
||||
}
|
||||
|
||||
if (isDone) break;
|
||||
}
|
||||
|
||||
return { ok: true, stats, cursor, isDone };
|
||||
}
|
||||
|
||||
export const backfillSkillCapabilityTagsInternal = internalAction({
|
||||
args: {
|
||||
dryRun: v.optional(v.boolean()),
|
||||
cursor: v.optional(v.string()),
|
||||
batchSize: v.optional(v.number()),
|
||||
maxBatches: v.optional(v.number()),
|
||||
delayMs: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<CapabilityBackfillResult> => {
|
||||
const result = await backfillSkillCapabilityTagsInternalHandler(ctx, args);
|
||||
|
||||
if (!args.dryRun && !result.isDone && result.cursor) {
|
||||
const delayMs = clampInt(args.delayMs ?? DEFAULT_CAPABILITY_BACKFILL_DELAY_MS, 0, 60_000);
|
||||
await ctx.scheduler.runAfter(
|
||||
delayMs,
|
||||
internal.maintenance.backfillSkillCapabilityTagsInternal,
|
||||
{
|
||||
dryRun: false,
|
||||
cursor: result.cursor,
|
||||
batchSize: args.batchSize,
|
||||
maxBatches: 1,
|
||||
delayMs,
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
return result;
|
||||
},
|
||||
});
|
||||
|
||||
export const backfillSkillCapabilityTags: ReturnType<typeof action> = action({
|
||||
args: {
|
||||
dryRun: v.optional(v.boolean()),
|
||||
cursor: v.optional(v.string()),
|
||||
batchSize: v.optional(v.number()),
|
||||
maxBatches: v.optional(v.number()),
|
||||
delayMs: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<CapabilityBackfillResult> => {
|
||||
const { user } = await requireUserFromAction(ctx);
|
||||
assertRole(user, ["admin"]);
|
||||
return ctx.runAction(internal.maintenance.backfillSkillCapabilityTagsInternal, args);
|
||||
},
|
||||
});
|
||||
|
||||
type FingerprintBackfillStats = {
|
||||
versionsScanned: number;
|
||||
versionsPatched: number;
|
||||
|
||||
@@ -0,0 +1,70 @@
|
||||
import { v } from "convex/values";
|
||||
import { internalMutation, internalQuery } from "./functions";
|
||||
|
||||
export const createInternal = internalMutation({
|
||||
args: {
|
||||
packageId: v.id("packages"),
|
||||
version: v.string(),
|
||||
prefix: v.string(),
|
||||
tokenHash: v.string(),
|
||||
provider: v.literal("github-actions"),
|
||||
repository: v.string(),
|
||||
repositoryId: v.string(),
|
||||
repositoryOwner: v.string(),
|
||||
repositoryOwnerId: v.string(),
|
||||
workflowFilename: v.string(),
|
||||
environment: v.optional(v.string()),
|
||||
runId: v.string(),
|
||||
runAttempt: v.string(),
|
||||
sha: v.string(),
|
||||
ref: v.string(),
|
||||
refType: v.optional(v.string()),
|
||||
actor: v.optional(v.string()),
|
||||
actorId: v.optional(v.string()),
|
||||
expiresAt: v.number(),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const now = Date.now();
|
||||
return await ctx.db.insert("packagePublishTokens", {
|
||||
...args,
|
||||
createdAt: now,
|
||||
lastUsedAt: undefined,
|
||||
revokedAt: undefined,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const getByHashInternal = internalQuery({
|
||||
args: { tokenHash: v.string() },
|
||||
handler: async (ctx, args) => {
|
||||
return await ctx.db
|
||||
.query("packagePublishTokens")
|
||||
.withIndex("by_hash", (q) => q.eq("tokenHash", args.tokenHash))
|
||||
.unique();
|
||||
},
|
||||
});
|
||||
|
||||
export const getByIdInternal = internalQuery({
|
||||
args: { tokenId: v.id("packagePublishTokens") },
|
||||
handler: async (ctx, args) => {
|
||||
return await ctx.db.get(args.tokenId);
|
||||
},
|
||||
});
|
||||
|
||||
export const touchInternal = internalMutation({
|
||||
args: { tokenId: v.id("packagePublishTokens") },
|
||||
handler: async (ctx, args) => {
|
||||
const token = await ctx.db.get(args.tokenId);
|
||||
if (!token || token.revokedAt || token.expiresAt <= Date.now()) return;
|
||||
await ctx.db.patch(token._id, { lastUsedAt: Date.now() });
|
||||
},
|
||||
});
|
||||
|
||||
export const revokeInternal = internalMutation({
|
||||
args: { tokenId: v.id("packagePublishTokens") },
|
||||
handler: async (ctx, args) => {
|
||||
const token = await ctx.db.get(args.tokenId);
|
||||
if (!token || token.revokedAt) return;
|
||||
await ctx.db.patch(token._id, { revokedAt: Date.now() });
|
||||
},
|
||||
});
|
||||
+1015
-23
File diff suppressed because it is too large
Load Diff
+1230
-329
File diff suppressed because it is too large
Load Diff
+514
-89
@@ -2,8 +2,10 @@ import { getAuthUserId } from "@convex-dev/auth/server";
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import {
|
||||
addMember,
|
||||
listMine,
|
||||
migrateLegacyPublisherHandleToOrgInternal,
|
||||
removeMember,
|
||||
updateProfile,
|
||||
} from "./publishers";
|
||||
|
||||
vi.mock("@convex-dev/auth/server", () => ({
|
||||
@@ -15,9 +17,11 @@ type WrappedHandler<TArgs, TResult = unknown> = {
|
||||
};
|
||||
|
||||
const addMemberHandler = (
|
||||
addMember as unknown as WrappedHandler<
|
||||
{ publisherId: string; userHandle: string; role: "owner" | "admin" | "publisher" }
|
||||
>
|
||||
addMember as unknown as WrappedHandler<{
|
||||
publisherId: string;
|
||||
userHandle: string;
|
||||
role: "owner" | "admin" | "publisher";
|
||||
}>
|
||||
)._handler;
|
||||
|
||||
const removeMemberHandler = (
|
||||
@@ -45,6 +49,19 @@ const migrateLegacyPublisherHandleToOrgInternalHandler = (
|
||||
>
|
||||
)._handler;
|
||||
|
||||
const listMineHandler = (
|
||||
listMine as unknown as WrappedHandler<Record<string, never>, Array<unknown>>
|
||||
)._handler;
|
||||
|
||||
const updateProfileHandler = (
|
||||
updateProfile as unknown as WrappedHandler<{
|
||||
publisherId: string;
|
||||
displayName: string;
|
||||
bio?: string;
|
||||
image?: string;
|
||||
}>
|
||||
)._handler;
|
||||
|
||||
describe("publishers membership controls", () => {
|
||||
it("prevents admins from promoting members to owner", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:admin" as never);
|
||||
@@ -164,6 +181,391 @@ describe("publishers membership controls", () => {
|
||||
),
|
||||
).rejects.toThrow("Publisher must have at least one owner");
|
||||
});
|
||||
|
||||
it("adds a member when the requested handle resolves via a personal publisher", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:owner" as never);
|
||||
const publisherMembers: Array<Record<string, unknown>> = [
|
||||
{
|
||||
_id: "publisherMembers:owner",
|
||||
publisherId: "publishers:org",
|
||||
userId: "users:owner",
|
||||
role: "owner",
|
||||
},
|
||||
];
|
||||
const insert = vi.fn(async (table: string, value: Record<string, unknown>) => {
|
||||
if (table === "publisherMembers") {
|
||||
const row = { _id: "publisherMembers:new", ...value };
|
||||
publisherMembers.push(row);
|
||||
return row._id;
|
||||
}
|
||||
if (table === "auditLogs") return "auditLogs:1";
|
||||
if (table === "publishers") return "publishers:jaredforreal";
|
||||
throw new Error(`unexpected insert ${table}`);
|
||||
});
|
||||
const ctx = {
|
||||
db: {
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:owner") return { _id: id };
|
||||
if (id === "users:jared") {
|
||||
return {
|
||||
_id: id,
|
||||
_creationTime: 1,
|
||||
handle: undefined,
|
||||
name: "JaredForReal",
|
||||
displayName: "Jared",
|
||||
trustedPublisher: false,
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
};
|
||||
}
|
||||
if (id === "publishers:org") {
|
||||
return {
|
||||
_id: id,
|
||||
kind: "org",
|
||||
handle: "zai-org",
|
||||
displayName: "ZAI Org",
|
||||
};
|
||||
}
|
||||
if (id === "publishers:jaredforreal") {
|
||||
return {
|
||||
_id: id,
|
||||
_creationTime: 1,
|
||||
kind: "user",
|
||||
handle: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
linkedUserId: "users:jared",
|
||||
trustedPublisher: false,
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
if (indexName !== "by_publisher_user") {
|
||||
throw new Error(`unexpected index ${indexName}`);
|
||||
}
|
||||
let publisherId = "";
|
||||
let userId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "publisherId") publisherId = value;
|
||||
if (field === "userId") userId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(
|
||||
async () =>
|
||||
publisherMembers.find(
|
||||
(member) =>
|
||||
member.publisherId === publisherId && member.userId === userId,
|
||||
) ?? null,
|
||||
),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
if (indexName !== "handle") {
|
||||
throw new Error(`unexpected index ${indexName}`);
|
||||
}
|
||||
let handle = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () => {
|
||||
if (handle === "owner") return { _id: "users:owner", handle: "owner" };
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
if (field === "linkedUserId") linkedUserId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () => {
|
||||
if (indexName === "by_handle" && handle === "jaredforreal") {
|
||||
return {
|
||||
_id: "publishers:jaredforreal",
|
||||
_creationTime: 1,
|
||||
kind: "user",
|
||||
handle: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
linkedUserId: "users:jared",
|
||||
trustedPublisher: false,
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
};
|
||||
}
|
||||
if (indexName === "by_linked_user" && linkedUserId === "users:jared") {
|
||||
return {
|
||||
_id: "publishers:jaredforreal",
|
||||
_creationTime: 1,
|
||||
kind: "user",
|
||||
handle: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
linkedUserId: "users:jared",
|
||||
trustedPublisher: false,
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
insert,
|
||||
patch: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
replace: vi.fn(),
|
||||
normalizeId: vi.fn(),
|
||||
},
|
||||
};
|
||||
|
||||
await expect(
|
||||
addMemberHandler(
|
||||
ctx as never,
|
||||
{ publisherId: "publishers:org", userHandle: "jaredforreal", role: "admin" } as never,
|
||||
),
|
||||
).resolves.toEqual({ ok: true });
|
||||
|
||||
expect(insert).toHaveBeenCalledWith(
|
||||
"publisherMembers",
|
||||
expect.objectContaining({
|
||||
publisherId: "publishers:org",
|
||||
userId: "users:jared",
|
||||
role: "admin",
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("lets org admins update org profile fields", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:admin" as never);
|
||||
const patch = vi.fn(async () => {});
|
||||
const insert = vi.fn(async () => "auditLogs:1");
|
||||
const ctx = {
|
||||
db: {
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:admin") return { _id: id };
|
||||
if (id === "publishers:org") {
|
||||
return {
|
||||
_id: id,
|
||||
kind: "org",
|
||||
handle: "shopify",
|
||||
displayName: "Shopify",
|
||||
image: undefined,
|
||||
bio: undefined,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: vi.fn(() => ({
|
||||
unique: vi.fn().mockResolvedValue({
|
||||
_id: "publisherMembers:admin",
|
||||
publisherId: "publishers:org",
|
||||
userId: "users:admin",
|
||||
role: "admin",
|
||||
}),
|
||||
})),
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
patch,
|
||||
insert,
|
||||
delete: vi.fn(),
|
||||
replace: vi.fn(),
|
||||
normalizeId: vi.fn(),
|
||||
},
|
||||
};
|
||||
|
||||
await expect(
|
||||
updateProfileHandler(
|
||||
ctx as never,
|
||||
{
|
||||
publisherId: "publishers:org",
|
||||
displayName: "Shopify",
|
||||
bio: "Commerce platform",
|
||||
image: "https://cdn.example.com/shopify.png",
|
||||
} as never,
|
||||
),
|
||||
).resolves.toEqual({
|
||||
ok: true,
|
||||
publisher: expect.objectContaining({
|
||||
_id: "publishers:org",
|
||||
displayName: "Shopify",
|
||||
}),
|
||||
});
|
||||
|
||||
expect(patch).toHaveBeenCalledWith(
|
||||
"publishers:org",
|
||||
expect.objectContaining({
|
||||
displayName: "Shopify",
|
||||
bio: "Commerce platform",
|
||||
image: "https://cdn.example.com/shopify.png",
|
||||
}),
|
||||
);
|
||||
expect(insert).toHaveBeenCalledWith(
|
||||
"auditLogs",
|
||||
expect.objectContaining({
|
||||
action: "publisher.profile.update",
|
||||
targetId: "publishers:org",
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("rejects invalid org profile image URLs", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:admin" as never);
|
||||
const ctx = {
|
||||
db: {
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:admin") return { _id: id };
|
||||
if (id === "publishers:org") {
|
||||
return {
|
||||
_id: id,
|
||||
kind: "org",
|
||||
handle: "shopify",
|
||||
displayName: "Shopify",
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: vi.fn(() => ({
|
||||
unique: vi.fn().mockResolvedValue({
|
||||
_id: "publisherMembers:admin",
|
||||
publisherId: "publishers:org",
|
||||
userId: "users:admin",
|
||||
role: "admin",
|
||||
}),
|
||||
})),
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
patch: vi.fn(),
|
||||
insert: vi.fn(),
|
||||
delete: vi.fn(),
|
||||
replace: vi.fn(),
|
||||
normalizeId: vi.fn(),
|
||||
},
|
||||
};
|
||||
|
||||
await expect(
|
||||
updateProfileHandler(
|
||||
ctx as never,
|
||||
{
|
||||
publisherId: "publishers:org",
|
||||
displayName: "Shopify",
|
||||
image: "not-a-url",
|
||||
} as never,
|
||||
),
|
||||
).rejects.toThrow("Image must be a valid URL");
|
||||
});
|
||||
});
|
||||
|
||||
describe("publisher bootstrap", () => {
|
||||
it("lists a synthesized personal publisher when membership rows are missing", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:alice" as never);
|
||||
const ctx = {
|
||||
db: {
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:alice") {
|
||||
return {
|
||||
_id: id,
|
||||
_creationTime: 1,
|
||||
handle: "alice",
|
||||
displayName: "Alice",
|
||||
trustedPublisher: false,
|
||||
createdAt: 1,
|
||||
updatedAt: 1,
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: vi.fn((indexName: string) => {
|
||||
if (indexName !== "by_user") throw new Error(`unexpected index ${indexName}`);
|
||||
return { collect: vi.fn().mockResolvedValue([]) };
|
||||
}),
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: vi.fn((indexName: string) => {
|
||||
if (indexName !== "by_linked_user") {
|
||||
throw new Error(`unexpected index ${indexName}`);
|
||||
}
|
||||
return { unique: vi.fn().mockResolvedValue(null) };
|
||||
}),
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
},
|
||||
};
|
||||
|
||||
await expect(listMineHandler(ctx as never, {} as never)).resolves.toEqual([
|
||||
expect.objectContaining({
|
||||
role: "owner",
|
||||
publisher: expect.objectContaining({
|
||||
handle: "alice",
|
||||
kind: "user",
|
||||
linkedUserId: "users:alice",
|
||||
}),
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("legacy publisher migration", () => {
|
||||
@@ -282,101 +684,126 @@ describe("legacy publisher migration", () => {
|
||||
const query = vi.fn((table: string) => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: vi.fn((_indexName: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
let handle = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () =>
|
||||
[...users.values()].find((user) => user.handle === handle) ?? null,
|
||||
),
|
||||
};
|
||||
}),
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
_indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(
|
||||
async () => [...users.values()].find((user) => user.handle === handle) ?? null,
|
||||
),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: vi.fn((_indexName: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
if (field === "linkedUserId") linkedUserId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () => {
|
||||
if (handle) {
|
||||
return [...publishers.values()].find((publisher) => publisher.handle === handle) ?? null;
|
||||
}
|
||||
if (linkedUserId) {
|
||||
return (
|
||||
[...publishers.values()].find((publisher) => publisher.linkedUserId === linkedUserId) ??
|
||||
null
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
}),
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
_indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
if (field === "linkedUserId") linkedUserId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () => {
|
||||
if (handle) {
|
||||
return (
|
||||
[...publishers.values()].find((publisher) => publisher.handle === handle) ??
|
||||
null
|
||||
);
|
||||
}
|
||||
if (linkedUserId) {
|
||||
return (
|
||||
[...publishers.values()].find(
|
||||
(publisher) => publisher.linkedUserId === linkedUserId,
|
||||
) ?? null
|
||||
);
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: vi.fn((_indexName: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
let publisherId = "";
|
||||
let userId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "publisherId") publisherId = value;
|
||||
if (field === "userId") userId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(async () =>
|
||||
publisherMembers.find(
|
||||
(member) => member.publisherId === publisherId && member.userId === userId,
|
||||
) ?? null,
|
||||
),
|
||||
};
|
||||
}),
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
_indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let publisherId = "";
|
||||
let userId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "publisherId") publisherId = value;
|
||||
if (field === "userId") userId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
unique: vi.fn(
|
||||
async () =>
|
||||
publisherMembers.find(
|
||||
(member) => member.publisherId === publisherId && member.userId === userId,
|
||||
) ?? null,
|
||||
),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "packages") {
|
||||
return {
|
||||
withIndex: vi.fn((_indexName: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
let ownerUserId = "";
|
||||
let ownerPublisherId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "ownerUserId") ownerUserId = value;
|
||||
if (field === "ownerPublisherId") ownerPublisherId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
collect: vi.fn(async () => {
|
||||
if (ownerUserId) {
|
||||
return packages.filter((pkg) => pkg.ownerUserId === ownerUserId);
|
||||
}
|
||||
if (ownerPublisherId) {
|
||||
return packages.filter((pkg) => pkg.ownerPublisherId === ownerPublisherId);
|
||||
}
|
||||
return [];
|
||||
}),
|
||||
};
|
||||
}),
|
||||
withIndex: vi.fn(
|
||||
(
|
||||
_indexName: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let ownerUserId = "";
|
||||
let ownerPublisherId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "ownerUserId") ownerUserId = value;
|
||||
if (field === "ownerPublisherId") ownerPublisherId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
return {
|
||||
collect: vi.fn(async () => {
|
||||
if (ownerUserId) {
|
||||
return packages.filter((pkg) => pkg.ownerUserId === ownerUserId);
|
||||
}
|
||||
if (ownerPublisherId) {
|
||||
return packages.filter((pkg) => pkg.ownerPublisherId === ownerPublisherId);
|
||||
}
|
||||
return [];
|
||||
}),
|
||||
};
|
||||
},
|
||||
),
|
||||
};
|
||||
}
|
||||
if (table === "skills") {
|
||||
@@ -392,9 +819,7 @@ describe("legacy publisher migration", () => {
|
||||
const result = await migrateLegacyPublisherHandleToOrgInternalHandler(
|
||||
{
|
||||
db: {
|
||||
get: vi.fn(async (id: string) =>
|
||||
users.get(id) ?? publishers.get(id) ?? null,
|
||||
),
|
||||
get: vi.fn(async (id: string) => users.get(id) ?? publishers.get(id) ?? null),
|
||||
query,
|
||||
patch,
|
||||
insert,
|
||||
|
||||
+94
-20
@@ -4,15 +4,17 @@ import type { Doc, Id } from "./_generated/dataModel";
|
||||
import type { MutationCtx } from "./_generated/server";
|
||||
import { internalMutation, internalQuery, mutation, query } from "./functions";
|
||||
import { assertAdmin, requireUser } from "./lib/access";
|
||||
import { toPublicPublisher } from "./lib/public";
|
||||
import {
|
||||
ensurePersonalPublisherForUser,
|
||||
getActiveUserByHandleOrPersonalPublisher,
|
||||
getPublisherByHandle,
|
||||
getPublisherMembership,
|
||||
getPersonalPublisherForUserOrFallback,
|
||||
getPersonalPublisherForUser,
|
||||
isPublisherRoleAllowed,
|
||||
normalizePublisherHandle,
|
||||
} from "./lib/publishers";
|
||||
import { toPublicPublisher } from "./lib/public";
|
||||
|
||||
const PUBLISHER_HANDLE_PATTERN = /^[a-z0-9](?:[a-z0-9-]{0,38}[a-z0-9])?$/;
|
||||
|
||||
@@ -86,10 +88,9 @@ async function migrateLegacyPublisherHandleToOrgWithActor(
|
||||
throw new ConvexError(`Legacy user "@${orgHandle}" not found`);
|
||||
}
|
||||
|
||||
const personalPublisher =
|
||||
legacyUser.personalPublisherId
|
||||
? await ctx.db.get(legacyUser.personalPublisherId)
|
||||
: await getPersonalPublisherForUser(ctx, legacyUser._id);
|
||||
const personalPublisher = legacyUser.personalPublisherId
|
||||
? await ctx.db.get(legacyUser.personalPublisherId)
|
||||
: await getPersonalPublisherForUser(ctx, legacyUser._id);
|
||||
const convertiblePublisher =
|
||||
handlePublisher?.kind === "user" && handlePublisher.linkedUserId === legacyUser._id
|
||||
? handlePublisher
|
||||
@@ -350,25 +351,22 @@ export const ensurePersonalPublisherInternal = internalMutation({
|
||||
},
|
||||
});
|
||||
|
||||
export const resolvePublishTargetForUserInternal = internalQuery({
|
||||
export const resolvePublishTargetForUserInternal = internalMutation({
|
||||
args: {
|
||||
actorUserId: v.id("users"),
|
||||
ownerHandle: v.optional(v.string()),
|
||||
minimumRole: v.optional(v.union(v.literal("owner"), v.literal("admin"), v.literal("publisher"))),
|
||||
minimumRole: v.optional(
|
||||
v.union(v.literal("owner"), v.literal("admin"), v.literal("publisher")),
|
||||
),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const actor = await ctx.db.get(args.actorUserId);
|
||||
if (!actor || actor.deletedAt || actor.deactivatedAt) throw new ConvexError("Unauthorized");
|
||||
const minimumRole = args.minimumRole ?? "publisher";
|
||||
const requestedHandle = normalizePublisherHandle(args.ownerHandle);
|
||||
const personal =
|
||||
actor.personalPublisherId
|
||||
? await ctx.db.get(actor.personalPublisherId)
|
||||
: await getPersonalPublisherForUser(ctx, actor._id);
|
||||
const personal = await ensurePersonalPublisherForUser(ctx, actor);
|
||||
if (!personal) throw new ConvexError("Personal publisher not found");
|
||||
if (!requestedHandle) {
|
||||
if (!personal || personal.deletedAt || personal.deactivatedAt) {
|
||||
throw new ConvexError("Personal publisher not found");
|
||||
}
|
||||
return {
|
||||
publisherId: personal._id,
|
||||
handle: personal.handle,
|
||||
@@ -408,6 +406,8 @@ export const listMine = query({
|
||||
handler: async (ctx) => {
|
||||
const userId = await getAuthUserId(ctx);
|
||||
if (!userId) return [];
|
||||
const user = await ctx.db.get(userId);
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return [];
|
||||
const memberships = await ctx.db
|
||||
.query("publisherMembers")
|
||||
.withIndex("by_user", (q) => q.eq("userId", userId))
|
||||
@@ -423,7 +423,7 @@ export const listMine = query({
|
||||
};
|
||||
}),
|
||||
);
|
||||
return publishers.filter(
|
||||
const visiblePublishers = publishers.filter(
|
||||
(
|
||||
item,
|
||||
): item is {
|
||||
@@ -431,6 +431,19 @@ export const listMine = query({
|
||||
role: Doc<"publisherMembers">["role"];
|
||||
} => Boolean(item),
|
||||
);
|
||||
const personalPublisher = toPublicPublisher(
|
||||
await getPersonalPublisherForUserOrFallback(ctx, user),
|
||||
);
|
||||
if (
|
||||
personalPublisher &&
|
||||
!visiblePublishers.some((entry) => entry.publisher._id === personalPublisher._id)
|
||||
) {
|
||||
visiblePublishers.unshift({
|
||||
publisher: personalPublisher,
|
||||
role: "owner",
|
||||
});
|
||||
}
|
||||
return visiblePublishers;
|
||||
},
|
||||
});
|
||||
|
||||
@@ -526,6 +539,70 @@ export const createOrg = mutation({
|
||||
},
|
||||
});
|
||||
|
||||
export const updateProfile = mutation({
|
||||
args: {
|
||||
publisherId: v.id("publishers"),
|
||||
displayName: v.string(),
|
||||
bio: v.optional(v.string()),
|
||||
image: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const { userId } = await requireUser(ctx);
|
||||
const publisher = await ctx.db.get(args.publisherId);
|
||||
if (!publisher || publisher.deletedAt || publisher.deactivatedAt) {
|
||||
throw new ConvexError("Publisher not found");
|
||||
}
|
||||
if (publisher.kind !== "org") {
|
||||
throw new ConvexError("Only org publishers can be updated here");
|
||||
}
|
||||
|
||||
const membership = await getPublisherMembership(ctx, publisher._id, userId);
|
||||
if (!membership || !isPublisherRoleAllowed(membership.role, ["admin"])) {
|
||||
throw new ConvexError("Forbidden");
|
||||
}
|
||||
|
||||
const displayName = args.displayName.trim() || publisher.handle;
|
||||
const bio = args.bio?.trim() || undefined;
|
||||
const image = args.image?.trim() || undefined;
|
||||
if (image) {
|
||||
let parsed: URL;
|
||||
try {
|
||||
parsed = new URL(image);
|
||||
} catch {
|
||||
throw new ConvexError("Image must be a valid URL");
|
||||
}
|
||||
if (parsed.protocol !== "http:" && parsed.protocol !== "https:") {
|
||||
throw new ConvexError("Image must use http or https");
|
||||
}
|
||||
}
|
||||
|
||||
const now = Date.now();
|
||||
await ctx.db.patch(publisher._id, {
|
||||
displayName,
|
||||
bio,
|
||||
image,
|
||||
updatedAt: now,
|
||||
});
|
||||
await ctx.db.insert("auditLogs", {
|
||||
actorUserId: userId,
|
||||
action: "publisher.profile.update",
|
||||
targetType: "publisher",
|
||||
targetId: publisher._id,
|
||||
metadata: {
|
||||
displayName,
|
||||
bio,
|
||||
image,
|
||||
},
|
||||
createdAt: now,
|
||||
});
|
||||
|
||||
return {
|
||||
ok: true as const,
|
||||
publisher: toPublicPublisher(await ctx.db.get(publisher._id)),
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const migrateLegacyPublisherHandleToOrg = mutation({
|
||||
args: {
|
||||
handle: v.string(),
|
||||
@@ -573,11 +650,8 @@ export const addMember = mutation({
|
||||
}
|
||||
const handle = normalizePublisherHandle(args.userHandle);
|
||||
if (!handle) throw new ConvexError("User handle is required");
|
||||
const targetUser = await ctx.db
|
||||
.query("users")
|
||||
.withIndex("handle", (q) => q.eq("handle", handle))
|
||||
.unique();
|
||||
if (!targetUser || targetUser.deletedAt || targetUser.deactivatedAt) {
|
||||
const targetUser = await getActiveUserByHandleOrPersonalPublisher(ctx, handle);
|
||||
if (!targetUser) {
|
||||
throw new ConvexError(`User "@${handle}" not found`);
|
||||
}
|
||||
await ensurePersonalPublisherForUser(ctx, targetUser);
|
||||
|
||||
+89
-8
@@ -28,6 +28,9 @@ const users = defineTable({
|
||||
githubFetchedAt: v.optional(v.number()),
|
||||
githubProfileSyncedAt: v.optional(v.number()),
|
||||
trustedPublisher: v.optional(v.boolean()),
|
||||
publishedSkills: v.optional(v.number()),
|
||||
totalStars: v.optional(v.number()),
|
||||
totalDownloads: v.optional(v.number()),
|
||||
personalPublisherId: v.optional(v.id("publishers")),
|
||||
requiresModerationAt: v.optional(v.number()),
|
||||
requiresModerationReason: v.optional(v.string()),
|
||||
@@ -40,7 +43,8 @@ const users = defineTable({
|
||||
})
|
||||
.index("email", ["email"])
|
||||
.index("phone", ["phone"])
|
||||
.index("handle", ["handle"]);
|
||||
.index("handle", ["handle"])
|
||||
.index("by_active_handle", ["deletedAt", "deactivatedAt", "handle"]);
|
||||
|
||||
const publishers = defineTable({
|
||||
kind: v.union(v.literal("user"), v.literal("org")),
|
||||
@@ -91,10 +95,22 @@ const badgesValidator = v.optional(
|
||||
}),
|
||||
);
|
||||
|
||||
/**
|
||||
* Nested stat fields on the `skills` document.
|
||||
*
|
||||
* The four migrated fields below are kept for backward compatibility only.
|
||||
* Always use the top-level fields (`statsDownloads`, `statsStars`,
|
||||
* `statsInstallsCurrent`, `statsInstallsAllTime`) as the source of truth,
|
||||
* and use `readCanonicalStat()` / `applySkillStatDeltas()` to read/write them.
|
||||
*/
|
||||
const statsValidator = v.object({
|
||||
/** @deprecated Use top-level `statsDownloads` instead. */
|
||||
downloads: v.number(),
|
||||
/** @deprecated Use top-level `statsInstallsCurrent` instead. */
|
||||
installsCurrent: v.optional(v.number()),
|
||||
/** @deprecated Use top-level `statsInstallsAllTime` instead. */
|
||||
installsAllTime: v.optional(v.number()),
|
||||
/** @deprecated Use top-level `statsStars` instead. */
|
||||
stars: v.number(),
|
||||
versions: v.number(),
|
||||
comments: v.number(),
|
||||
@@ -188,6 +204,23 @@ const packageVerificationValidator = v.optional(
|
||||
}),
|
||||
);
|
||||
|
||||
const packagePublishActorValidator = v.optional(
|
||||
v.union(
|
||||
v.object({
|
||||
kind: v.literal("user"),
|
||||
userId: v.id("users"),
|
||||
}),
|
||||
v.object({
|
||||
kind: v.literal("github-actions"),
|
||||
repository: v.string(),
|
||||
workflow: v.string(),
|
||||
runId: v.string(),
|
||||
runAttempt: v.string(),
|
||||
sha: v.string(),
|
||||
}),
|
||||
),
|
||||
);
|
||||
|
||||
const packageScanStatusValidator = v.optional(
|
||||
v.union(
|
||||
v.literal("clean"),
|
||||
@@ -228,6 +261,7 @@ const skills = defineTable({
|
||||
}),
|
||||
),
|
||||
tags: v.record(v.string(), v.id("skillVersions")),
|
||||
capabilityTags: v.optional(v.array(v.string())),
|
||||
softDeletedAt: v.optional(v.number()),
|
||||
badges: badgesValidator,
|
||||
moderationStatus: moderationStatusValidator,
|
||||
@@ -366,7 +400,8 @@ const souls = defineTable({
|
||||
.index("by_slug", ["slug"])
|
||||
.index("by_owner", ["ownerUserId"])
|
||||
.index("by_owner_publisher", ["ownerPublisherId"])
|
||||
.index("by_updated", ["updatedAt"]);
|
||||
.index("by_updated", ["updatedAt"])
|
||||
.index("by_active_updated", ["softDeletedAt", "updatedAt"]);
|
||||
|
||||
const skillVersions = defineTable({
|
||||
skillId: v.id("skills"),
|
||||
@@ -425,6 +460,7 @@ const skillVersions = defineTable({
|
||||
checkedAt: v.number(),
|
||||
}),
|
||||
),
|
||||
capabilityTags: v.optional(v.array(v.string())),
|
||||
staticScan: v.optional(
|
||||
v.object({
|
||||
status: v.union(v.literal("clean"), v.literal("suspicious"), v.literal("malicious")),
|
||||
@@ -566,6 +602,7 @@ const skillSearchDigest = defineTable({
|
||||
}),
|
||||
),
|
||||
tags: v.record(v.string(), v.id("skillVersions")),
|
||||
capabilityTags: v.optional(v.array(v.string())),
|
||||
badges: badgesValidator,
|
||||
stats: statsValidator,
|
||||
statsDownloads: v.optional(v.number()),
|
||||
@@ -719,6 +756,7 @@ const packageReleases = defineTable({
|
||||
),
|
||||
source: v.optional(v.any()),
|
||||
createdBy: v.id("users"),
|
||||
publishActor: packagePublishActorValidator,
|
||||
createdAt: v.number(),
|
||||
softDeletedAt: v.optional(v.number()),
|
||||
})
|
||||
@@ -727,6 +765,50 @@ const packageReleases = defineTable({
|
||||
.index("by_package_version", ["packageId", "version"])
|
||||
.index("by_sha256hash", ["sha256hash"]);
|
||||
|
||||
const packageTrustedPublishers = defineTable({
|
||||
packageId: v.id("packages"),
|
||||
provider: v.literal("github-actions"),
|
||||
repository: v.string(),
|
||||
repositoryId: v.string(),
|
||||
repositoryOwner: v.string(),
|
||||
repositoryOwnerId: v.string(),
|
||||
workflowFilename: v.string(),
|
||||
environment: v.optional(v.string()),
|
||||
createdByUserId: v.id("users"),
|
||||
updatedByUserId: v.id("users"),
|
||||
createdAt: v.number(),
|
||||
updatedAt: v.number(),
|
||||
})
|
||||
.index("by_package", ["packageId"])
|
||||
.index("by_repository", ["repository", "workflowFilename"]);
|
||||
|
||||
const packagePublishTokens = defineTable({
|
||||
packageId: v.id("packages"),
|
||||
version: v.string(),
|
||||
prefix: v.string(),
|
||||
tokenHash: v.string(),
|
||||
provider: v.literal("github-actions"),
|
||||
repository: v.string(),
|
||||
repositoryId: v.string(),
|
||||
repositoryOwner: v.string(),
|
||||
repositoryOwnerId: v.string(),
|
||||
workflowFilename: v.string(),
|
||||
environment: v.optional(v.string()),
|
||||
runId: v.string(),
|
||||
runAttempt: v.string(),
|
||||
sha: v.string(),
|
||||
ref: v.string(),
|
||||
refType: v.optional(v.string()),
|
||||
actor: v.optional(v.string()),
|
||||
actorId: v.optional(v.string()),
|
||||
expiresAt: v.number(),
|
||||
lastUsedAt: v.optional(v.number()),
|
||||
revokedAt: v.optional(v.number()),
|
||||
createdAt: v.number(),
|
||||
})
|
||||
.index("by_hash", ["tokenHash"])
|
||||
.index("by_package", ["packageId", "version", "createdAt"]);
|
||||
|
||||
const packageSearchDigest = defineTable({
|
||||
packageId: v.id("packages"),
|
||||
name: v.string(),
|
||||
@@ -808,6 +890,8 @@ const packageSearchDigest = defineTable({
|
||||
"executesCode",
|
||||
"updatedAt",
|
||||
])
|
||||
.index("by_active_normalized_name", ["softDeletedAt", "normalizedName", "updatedAt"])
|
||||
.index("by_active_runtime_id", ["softDeletedAt", "runtimeId", "updatedAt"])
|
||||
.index("by_active_name", ["softDeletedAt", "displayName"]);
|
||||
|
||||
const packageCapabilitySearchDigest = defineTable({
|
||||
@@ -842,12 +926,7 @@ const packageCapabilitySearchDigest = defineTable({
|
||||
"executesCode",
|
||||
"updatedAt",
|
||||
])
|
||||
.index("by_active_family_tag_updated", [
|
||||
"softDeletedAt",
|
||||
"family",
|
||||
"capabilityTag",
|
||||
"updatedAt",
|
||||
])
|
||||
.index("by_active_family_tag_updated", ["softDeletedAt", "family", "capabilityTag", "updatedAt"])
|
||||
.index("by_active_family_tag_executes_updated", [
|
||||
"softDeletedAt",
|
||||
"family",
|
||||
@@ -1253,6 +1332,8 @@ export default defineSchema({
|
||||
skillSlugAliases,
|
||||
packages,
|
||||
packageReleases,
|
||||
packageTrustedPublishers,
|
||||
packagePublishTokens,
|
||||
packageSearchDigest,
|
||||
packageCapabilitySearchDigest,
|
||||
souls,
|
||||
|
||||
+292
-3
@@ -46,8 +46,8 @@ describe("search helpers", () => {
|
||||
owner: null,
|
||||
},
|
||||
];
|
||||
// With incremental hydration, empty vector results skip the hydrate call entirely.
|
||||
const runQuery = vi.fn().mockResolvedValueOnce(fallback); // lexicalFallbackSkills (only call)
|
||||
// Slug-like queries now do an indexed exact-slug lookup before lexical fallback.
|
||||
const runQuery = vi.fn().mockResolvedValueOnce(null).mockResolvedValueOnce(fallback);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
@@ -183,6 +183,7 @@ describe("search helpers", () => {
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(null) // getExactSkillSlugMatch
|
||||
.mockResolvedValueOnce(vectorEntries) // hydrateResults
|
||||
.mockResolvedValueOnce(fallbackEntries); // lexicalFallbackSkills
|
||||
|
||||
@@ -204,6 +205,289 @@ describe("search helpers", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("always includes an exact slug match even when vector exact matches already fill the limit", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const vectorEntries = Array.from({ length: 10 }, (_, index) => ({
|
||||
embeddingId: `skillEmbeddings:${index}`,
|
||||
skill: makePublicSkill({
|
||||
id: `skills:${index}`,
|
||||
slug: `downloader-${index}`,
|
||||
displayName: `Downloader ${index}`,
|
||||
downloads: 100 - index,
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
}));
|
||||
|
||||
const exactSlugEntry = {
|
||||
skill: makePublicSkill({
|
||||
id: "skills:exact",
|
||||
slug: "skill-downloader",
|
||||
displayName: "Skill Downloader",
|
||||
downloads: 1,
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "yyang100",
|
||||
owner: null,
|
||||
};
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(exactSlugEntry)
|
||||
.mockResolvedValueOnce(vectorEntries);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi
|
||||
.fn()
|
||||
.mockResolvedValue(
|
||||
vectorEntries.map((entry, index) => ({
|
||||
_id: entry.embeddingId,
|
||||
_score: 0.9 - index * 0.01,
|
||||
})),
|
||||
),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "skill-downloader", limit: 10 },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(10);
|
||||
expect(result[0].skill.slug).toBe("skill-downloader");
|
||||
expect(runQuery).toHaveBeenCalledTimes(2);
|
||||
});
|
||||
|
||||
it("omits exact slug injection when nonSuspiciousOnly excludes it", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const vectorEntries = [
|
||||
{
|
||||
embeddingId: "skillEmbeddings:1",
|
||||
skill: makePublicSkill({
|
||||
id: "skills:1",
|
||||
slug: "downloader-1",
|
||||
displayName: "Downloader 1",
|
||||
downloads: 50,
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
},
|
||||
];
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(null)
|
||||
.mockResolvedValueOnce(vectorEntries)
|
||||
.mockResolvedValueOnce([]);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi.fn().mockResolvedValue([{ _id: "skillEmbeddings:1", _score: 0.9 }]),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "skill-downloader", limit: 10, nonSuspiciousOnly: true },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].skill.slug).toBe("downloader-1");
|
||||
});
|
||||
|
||||
it("omits exact slug injection when highlightedOnly excludes it", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const exactSlugEntry = {
|
||||
skill: makePublicSkill({
|
||||
id: "skills:exact",
|
||||
slug: "skill-downloader",
|
||||
displayName: "Skill Downloader",
|
||||
downloads: 1,
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "yyang100",
|
||||
owner: null,
|
||||
};
|
||||
|
||||
const vectorEntries = [
|
||||
{
|
||||
embeddingId: "skillEmbeddings:1",
|
||||
skill: {
|
||||
...makePublicSkill({
|
||||
id: "skills:1",
|
||||
slug: "downloader-1",
|
||||
displayName: "Downloader 1",
|
||||
downloads: 50,
|
||||
}),
|
||||
badges: { highlighted: { byUserId: "users:mod", at: 1 } },
|
||||
},
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
},
|
||||
];
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(exactSlugEntry)
|
||||
.mockResolvedValueOnce(vectorEntries)
|
||||
.mockResolvedValueOnce([]);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi.fn().mockResolvedValue([{ _id: "skillEmbeddings:1", _score: 0.9 }]),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "skill-downloader", limit: 10, highlightedOnly: true },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].skill.slug).toBe("downloader-1");
|
||||
});
|
||||
|
||||
it("filters vector search results by capability tag", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(null)
|
||||
.mockResolvedValueOnce([
|
||||
{
|
||||
embeddingId: "skillEmbeddings:crypto",
|
||||
skill: makePublicSkill({
|
||||
id: "skills:crypto",
|
||||
slug: "wallet-helper",
|
||||
displayName: "Wallet Helper",
|
||||
capabilityTags: ["crypto", "requires-wallet"],
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
},
|
||||
{
|
||||
embeddingId: "skillEmbeddings:oauth",
|
||||
skill: makePublicSkill({
|
||||
id: "skills:oauth",
|
||||
slug: "x-poster",
|
||||
displayName: "X Poster",
|
||||
capabilityTags: ["requires-oauth-token", "posts-externally"],
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
},
|
||||
])
|
||||
.mockResolvedValueOnce([]);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi.fn().mockResolvedValue([
|
||||
{ _id: "skillEmbeddings:crypto", _score: 0.9 },
|
||||
{ _id: "skillEmbeddings:oauth", _score: 0.8 },
|
||||
]),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "helper", limit: 10, capabilityTag: "crypto" },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].skill.slug).toBe("wallet-helper");
|
||||
});
|
||||
|
||||
it("deduplicates exact slug injection against vector exact matches", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const sharedSkill = makePublicSkill({
|
||||
id: "skills:exact",
|
||||
slug: "skill-downloader",
|
||||
displayName: "Skill Downloader",
|
||||
downloads: 100,
|
||||
});
|
||||
const exactSlugEntry = {
|
||||
skill: sharedSkill,
|
||||
version: null,
|
||||
ownerHandle: "yyang100",
|
||||
owner: null,
|
||||
};
|
||||
const vectorEntries = [
|
||||
{
|
||||
embeddingId: "skillEmbeddings:exact",
|
||||
skill: sharedSkill,
|
||||
version: null,
|
||||
ownerHandle: "yyang100",
|
||||
owner: null,
|
||||
},
|
||||
{
|
||||
embeddingId: "skillEmbeddings:other",
|
||||
skill: makePublicSkill({
|
||||
id: "skills:other",
|
||||
slug: "downloader-2",
|
||||
displayName: "Downloader 2",
|
||||
downloads: 50,
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "owner",
|
||||
owner: null,
|
||||
},
|
||||
];
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(exactSlugEntry)
|
||||
.mockResolvedValueOnce(vectorEntries)
|
||||
.mockResolvedValueOnce([]);
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi.fn().mockResolvedValue([
|
||||
{ _id: "skillEmbeddings:exact", _score: 0.95 },
|
||||
{ _id: "skillEmbeddings:other", _score: 0.8 },
|
||||
]),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "skill-downloader", limit: 10 },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(2);
|
||||
expect(result.filter((entry) => entry.skill._id === "skills:exact")).toHaveLength(1);
|
||||
});
|
||||
|
||||
it("skips duplicate slug lookup inside lexical fallback when search action already did it", async () => {
|
||||
generateEmbeddingMock.mockResolvedValueOnce([0, 1, 2]);
|
||||
|
||||
const fallbackEntries = [
|
||||
{
|
||||
skill: makePublicSkill({
|
||||
id: "skills:orf",
|
||||
slug: "orf",
|
||||
displayName: "ORF",
|
||||
}),
|
||||
version: null,
|
||||
ownerHandle: "steipete",
|
||||
owner: null,
|
||||
},
|
||||
];
|
||||
|
||||
const runQuery = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(null)
|
||||
.mockImplementationOnce(async (_ref: unknown, args: { skipExactSlugLookup?: boolean }) => {
|
||||
expect(args.skipExactSlugLookup).toBe(true);
|
||||
return fallbackEntries;
|
||||
});
|
||||
|
||||
const result = await searchSkillsHandler(
|
||||
{
|
||||
vectorSearch: vi.fn().mockResolvedValue([]),
|
||||
runQuery,
|
||||
},
|
||||
{ query: "orf", limit: 10 },
|
||||
);
|
||||
|
||||
expect(result).toHaveLength(1);
|
||||
expect(result[0].skill.slug).toBe("orf");
|
||||
});
|
||||
|
||||
it("filters suspicious vector results in hydrateResults when requested", async () => {
|
||||
const result = await hydrateResultsHandler(
|
||||
{
|
||||
@@ -525,7 +809,10 @@ describe("search helpers", () => {
|
||||
|
||||
const hydrateCalls: string[][] = [];
|
||||
const runQuery = vi.fn(
|
||||
async (_ref: unknown, args: { embeddingIds?: string[]; query?: string }) => {
|
||||
async (_ref: unknown, args: { embeddingIds?: string[]; query?: string; slug?: string }) => {
|
||||
if (args.slug) {
|
||||
return null; // getExactSkillSlugMatch
|
||||
}
|
||||
if (args.embeddingIds) {
|
||||
hydrateCalls.push(args.embeddingIds);
|
||||
return args.embeddingIds.map((embeddingId: string) => ({
|
||||
@@ -586,6 +873,7 @@ function makePublicSkill(params: {
|
||||
slug: string;
|
||||
displayName: string;
|
||||
downloads?: number;
|
||||
capabilityTags?: string[];
|
||||
}) {
|
||||
return {
|
||||
_id: params.id,
|
||||
@@ -598,6 +886,7 @@ function makePublicSkill(params: {
|
||||
forkOf: undefined,
|
||||
latestVersionId: "skillVersions:1",
|
||||
tags: {},
|
||||
capabilityTags: params.capabilityTags,
|
||||
badges: {},
|
||||
stats: {
|
||||
downloads: params.downloads ?? 0,
|
||||
|
||||
+75
-8
@@ -7,6 +7,7 @@ import { isSkillHighlighted } from "./lib/badges";
|
||||
import { generateEmbedding } from "./lib/embeddings";
|
||||
import type { HydratableSkill, PublicPublisher } from "./lib/public";
|
||||
import { toPublicPublisher, toPublicSkill, toPublicSoul } from "./lib/public";
|
||||
import { SKILL_CAPABILITY_TAGS } from "./lib/skillCapabilityTags";
|
||||
import { getOwnerPublisher } from "./lib/publishers";
|
||||
import { matchesExactTokens, tokenize } from "./lib/searchText";
|
||||
import { isSkillSuspicious } from "./lib/skillSafety";
|
||||
@@ -51,6 +52,7 @@ const NAME_EXACT_BOOST = 1.1;
|
||||
const NAME_PREFIX_BOOST = 0.6;
|
||||
const POPULARITY_WEIGHT = 0.08;
|
||||
const FALLBACK_SCAN_LIMIT = 500;
|
||||
const SKILL_CAPABILITY_TAG_SET = new Set<string>(SKILL_CAPABILITY_TAGS);
|
||||
|
||||
function getNextCandidateLimit(current: number, max: number) {
|
||||
const next = Math.min(current * 2, max);
|
||||
@@ -116,18 +118,44 @@ function mergeUniqueBySkillId(primary: SkillSearchEntry[], fallback: SkillSearch
|
||||
return out;
|
||||
}
|
||||
|
||||
function isSlugLikeQuery(query: string) {
|
||||
return /^[a-z0-9][a-z0-9-]*$/.test(query.trim().toLowerCase());
|
||||
}
|
||||
|
||||
function matchesCapabilityTag(
|
||||
skill: Pick<HydratableSkill, "capabilityTags">,
|
||||
capabilityTag?: string,
|
||||
) {
|
||||
if (!capabilityTag) return true;
|
||||
return (skill.capabilityTags ?? []).includes(capabilityTag);
|
||||
}
|
||||
|
||||
export const searchSkills: ReturnType<typeof action> = action({
|
||||
args: {
|
||||
query: v.string(),
|
||||
limit: v.optional(v.number()),
|
||||
highlightedOnly: v.optional(v.boolean()),
|
||||
nonSuspiciousOnly: v.optional(v.boolean()),
|
||||
capabilityTag: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<SearchResult[]> => {
|
||||
const query = args.query.trim();
|
||||
if (!query) return [];
|
||||
if (args.capabilityTag && !SKILL_CAPABILITY_TAG_SET.has(args.capabilityTag)) return [];
|
||||
const queryTokens = tokenize(query);
|
||||
if (queryTokens.length === 0) return [];
|
||||
const rawExactSlugMatch = isSlugLikeQuery(query)
|
||||
? ((await ctx.runQuery(internal.search.getExactSkillSlugMatch, {
|
||||
slug: query.toLowerCase(),
|
||||
nonSuspiciousOnly: args.nonSuspiciousOnly,
|
||||
})) as SkillSearchEntry | null)
|
||||
: null;
|
||||
const exactSlugMatch =
|
||||
rawExactSlugMatch &&
|
||||
(!args.highlightedOnly || isSkillHighlighted(rawExactSlugMatch.skill)) &&
|
||||
matchesCapabilityTag(rawExactSlugMatch.skill, args.capabilityTag)
|
||||
? rawExactSlugMatch
|
||||
: null;
|
||||
let vector: number[];
|
||||
try {
|
||||
vector = await generateEmbedding(query);
|
||||
@@ -171,9 +199,11 @@ export const searchSkills: ReturnType<typeof action> = action({
|
||||
|
||||
// Skills already have badges from their docs (via toPublicSkill).
|
||||
// No need for a separate badge table lookup.
|
||||
const filtered = args.highlightedOnly
|
||||
? hydrated.filter((entry) => isSkillHighlighted(entry.skill))
|
||||
: hydrated;
|
||||
const filtered = hydrated.filter(
|
||||
(entry) =>
|
||||
(!args.highlightedOnly || isSkillHighlighted(entry.skill)) &&
|
||||
matchesCapabilityTag(entry.skill, args.capabilityTag),
|
||||
);
|
||||
|
||||
exactMatches = filtered.filter((entry) =>
|
||||
matchesExactTokens(queryTokens, [
|
||||
@@ -192,8 +222,12 @@ export const searchSkills: ReturnType<typeof action> = action({
|
||||
candidateLimit = nextLimit;
|
||||
}
|
||||
|
||||
const primaryMatches = exactSlugMatch
|
||||
? mergeUniqueBySkillId([exactSlugMatch], exactMatches)
|
||||
: exactMatches;
|
||||
|
||||
const fallbackMatches =
|
||||
exactMatches.length >= limit
|
||||
primaryMatches.length >= limit
|
||||
? []
|
||||
: ((await ctx.runQuery(internal.search.lexicalFallbackSkills, {
|
||||
query,
|
||||
@@ -201,9 +235,10 @@ export const searchSkills: ReturnType<typeof action> = action({
|
||||
limit: Math.min(Math.max(limit * 4, 200), FALLBACK_SCAN_LIMIT),
|
||||
highlightedOnly: args.highlightedOnly,
|
||||
nonSuspiciousOnly: args.nonSuspiciousOnly,
|
||||
capabilityTag: args.capabilityTag,
|
||||
skipExactSlugLookup: true,
|
||||
})) as SkillSearchEntry[]);
|
||||
|
||||
const mergedMatches = mergeUniqueBySkillId(exactMatches, fallbackMatches);
|
||||
const mergedMatches = mergeUniqueBySkillId(primaryMatches, fallbackMatches);
|
||||
|
||||
return mergedMatches
|
||||
.map((entry) => {
|
||||
@@ -225,6 +260,33 @@ export const searchSkills: ReturnType<typeof action> = action({
|
||||
},
|
||||
});
|
||||
|
||||
export const getExactSkillSlugMatch = internalQuery({
|
||||
args: {
|
||||
slug: v.string(),
|
||||
nonSuspiciousOnly: v.optional(v.boolean()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<SkillSearchEntry | null> => {
|
||||
const skill = await ctx.db
|
||||
.query("skills")
|
||||
.withIndex("by_slug", (q) => q.eq("slug", args.slug))
|
||||
.unique();
|
||||
if (!skill || skill.softDeletedAt) return null;
|
||||
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) return null;
|
||||
|
||||
const getOwnerInfo = makeOwnerInfoGetter(ctx);
|
||||
const resolved = await getOwnerInfo(skill.ownerUserId, skill.ownerPublisherId);
|
||||
const publicSkill = toPublicSkill(skill);
|
||||
if (!publicSkill || !resolved.owner) return null;
|
||||
|
||||
return {
|
||||
skill: publicSkill,
|
||||
version: null,
|
||||
ownerHandle: resolved.ownerHandle,
|
||||
owner: resolved.owner,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const hydrateResults = internalQuery({
|
||||
args: {
|
||||
embeddingIds: v.array(v.id("skillEmbeddings")),
|
||||
@@ -285,8 +347,11 @@ export const lexicalFallbackSkills = internalQuery({
|
||||
limit: v.optional(v.number()),
|
||||
highlightedOnly: v.optional(v.boolean()),
|
||||
nonSuspiciousOnly: v.optional(v.boolean()),
|
||||
capabilityTag: v.optional(v.string()),
|
||||
skipExactSlugLookup: v.optional(v.boolean()),
|
||||
},
|
||||
handler: async (ctx, args): Promise<SkillSearchEntry[]> => {
|
||||
if (args.capabilityTag && !SKILL_CAPABILITY_TAG_SET.has(args.capabilityTag)) return [];
|
||||
const limit = Math.min(Math.max(args.limit ?? 200, 10), FALLBACK_SCAN_LIMIT);
|
||||
const seenSkillIds = new Set<Id<"skills">>();
|
||||
const candidates: HydratableSkill[] = [];
|
||||
@@ -298,7 +363,7 @@ export const lexicalFallbackSkills = internalQuery({
|
||||
|
||||
// Exact slug match via the skills table (only one row, cheap).
|
||||
const slugQuery = args.query.trim().toLowerCase();
|
||||
if (/^[a-z0-9][a-z0-9-]*$/.test(slugQuery)) {
|
||||
if (!args.skipExactSlugLookup && /^[a-z0-9][a-z0-9-]*$/.test(slugQuery)) {
|
||||
const exactSlugSkill = await ctx.db
|
||||
.query("skills")
|
||||
.withIndex("by_slug", (q) => q.eq("slug", slugQuery))
|
||||
@@ -306,7 +371,8 @@ export const lexicalFallbackSkills = internalQuery({
|
||||
if (
|
||||
exactSlugSkill &&
|
||||
!exactSlugSkill.softDeletedAt &&
|
||||
(!args.nonSuspiciousOnly || !isSkillSuspicious(exactSlugSkill))
|
||||
(!args.nonSuspiciousOnly || !isSkillSuspicious(exactSlugSkill)) &&
|
||||
matchesCapabilityTag(exactSlugSkill, args.capabilityTag)
|
||||
) {
|
||||
seenSkillIds.add(exactSlugSkill._id);
|
||||
candidates.push(exactSlugSkill);
|
||||
@@ -324,6 +390,7 @@ export const lexicalFallbackSkills = internalQuery({
|
||||
if (seenSkillIds.has(digest.skillId)) continue;
|
||||
const skill = digestToHydratableSkill(digest);
|
||||
if (args.nonSuspiciousOnly && isSkillSuspicious(skill)) continue;
|
||||
if (!matchesCapabilityTag(skill, args.capabilityTag)) continue;
|
||||
seenSkillIds.add(digest.skillId);
|
||||
candidates.push(skill);
|
||||
// Pre-resolve owner from digest to avoid users table reads.
|
||||
|
||||
@@ -23,6 +23,7 @@ import type { Doc, Id } from "./_generated/dataModel";
|
||||
import type { MutationCtx } from "./_generated/server";
|
||||
import { internalAction, internalMutation, internalQuery } from "./functions";
|
||||
import { applySkillStatDeltas, bumpDailySkillStats } from "./lib/skillStats";
|
||||
import { adjustUserSkillStatsForSkillChange } from "./lib/userSkillStats";
|
||||
|
||||
/**
|
||||
* Event types that affect skill stats:
|
||||
@@ -259,6 +260,7 @@ export const processSkillStatEventsInternal = internalMutation({
|
||||
// Don't update `updatedAt` — stat changes shouldn't move the
|
||||
// skill's position in the by_active_updated index.
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, { ...skill, ...patch });
|
||||
}
|
||||
|
||||
// NOTE: Daily stats (skillDailyStats) are written by the 15-minute
|
||||
|
||||
@@ -61,7 +61,7 @@ describe("skillTransfers", () => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: () => ({
|
||||
first: async () => ({ _id: "users:2", handle: "alice", displayName: "Alice" }),
|
||||
unique: async () => ({ _id: "users:2", handle: "alice", displayName: "Alice" }),
|
||||
}),
|
||||
};
|
||||
}
|
||||
@@ -102,6 +102,242 @@ describe("skillTransfers", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("requestTransferInternal resolves recipient via personal publisher handle", async () => {
|
||||
const insert = vi.fn(async (table: string) => {
|
||||
if (table === "skillOwnershipTransfers") return "skillOwnershipTransfers:new";
|
||||
return "auditLogs:1";
|
||||
});
|
||||
|
||||
const result = (await requestTransferInternalHandler(
|
||||
{
|
||||
db: {
|
||||
normalizeId: vi.fn(),
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:1") return { _id: "users:1", handle: "owner" };
|
||||
if (id === "users:2") {
|
||||
return {
|
||||
_id: "users:2",
|
||||
handle: undefined,
|
||||
name: "Alice",
|
||||
displayName: "Alice",
|
||||
};
|
||||
}
|
||||
if (id === "skills:1") {
|
||||
return {
|
||||
_id: "skills:1",
|
||||
slug: "demo",
|
||||
displayName: "Demo",
|
||||
ownerUserId: "users:1",
|
||||
};
|
||||
}
|
||||
if (id === "publishers:alice") {
|
||||
return {
|
||||
_id: "publishers:alice",
|
||||
kind: "user",
|
||||
handle: "alice",
|
||||
displayName: "Alice",
|
||||
linkedUserId: "users:2",
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: () => ({
|
||||
unique: async () => null,
|
||||
}),
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: () => ({
|
||||
unique: async () => ({
|
||||
_id: "publishers:alice",
|
||||
kind: "user",
|
||||
handle: "alice",
|
||||
displayName: "Alice",
|
||||
linkedUserId: "users:2",
|
||||
}),
|
||||
}),
|
||||
};
|
||||
}
|
||||
if (table === "skillOwnershipTransfers") {
|
||||
return {
|
||||
withIndex: () => ({
|
||||
collect: async () => [],
|
||||
}),
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
patch: vi.fn(async () => {}),
|
||||
insert,
|
||||
},
|
||||
} as never,
|
||||
{
|
||||
actorUserId: "users:1",
|
||||
skillId: "skills:1",
|
||||
toUserHandle: "@alice",
|
||||
} as never,
|
||||
)) as { ok: boolean; transferId: string };
|
||||
|
||||
expect(result).toEqual(
|
||||
expect.objectContaining({
|
||||
ok: true,
|
||||
transferId: "skillOwnershipTransfers:new",
|
||||
toUserHandle: "alice",
|
||||
}),
|
||||
);
|
||||
expect(insert).toHaveBeenCalledWith(
|
||||
"skillOwnershipTransfers",
|
||||
expect.objectContaining({
|
||||
toUserId: "users:2",
|
||||
}),
|
||||
);
|
||||
});
|
||||
|
||||
it("acceptTransferInternal updates skill and alias ownership to the recipient publisher", async () => {
|
||||
const patch = vi.fn(async () => {});
|
||||
const insert = vi.fn(async () => "auditLogs:1");
|
||||
const newPublisher = {
|
||||
_id: "publishers:alice",
|
||||
handle: "alice",
|
||||
displayName: "Alice",
|
||||
linkedUserId: "users:2",
|
||||
trustedPublisher: false,
|
||||
};
|
||||
const existingMember = {
|
||||
_id: "publisherMembers:1",
|
||||
publisherId: "publishers:alice",
|
||||
userId: "users:2",
|
||||
role: "owner",
|
||||
};
|
||||
const aliases = [
|
||||
{
|
||||
_id: "skillSlugAliases:1",
|
||||
slug: "demo-old",
|
||||
skillId: "skills:1",
|
||||
ownerUserId: "users:1",
|
||||
ownerPublisherId: "publishers:owner",
|
||||
},
|
||||
{
|
||||
_id: "skillSlugAliases:2",
|
||||
slug: "demo-legacy",
|
||||
skillId: "skills:1",
|
||||
ownerUserId: "users:1",
|
||||
ownerPublisherId: "publishers:owner",
|
||||
},
|
||||
];
|
||||
|
||||
const result = (await acceptTransferInternalHandler(
|
||||
{
|
||||
db: {
|
||||
normalizeId: vi.fn(),
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === "users:2") {
|
||||
return {
|
||||
_id: "users:2",
|
||||
handle: "alice",
|
||||
personalPublisherId: "publishers:alice",
|
||||
trustedPublisher: false,
|
||||
};
|
||||
}
|
||||
if (id === "skillOwnershipTransfers:1") {
|
||||
return {
|
||||
_id: "skillOwnershipTransfers:1",
|
||||
skillId: "skills:1",
|
||||
fromUserId: "users:1",
|
||||
toUserId: "users:2",
|
||||
status: "pending",
|
||||
requestedAt: Date.now() - 1_000,
|
||||
expiresAt: Date.now() + 10_000,
|
||||
};
|
||||
}
|
||||
if (id === "skills:1") {
|
||||
return {
|
||||
_id: "skills:1",
|
||||
slug: "demo",
|
||||
ownerUserId: "users:1",
|
||||
ownerPublisherId: "publishers:owner",
|
||||
};
|
||||
}
|
||||
if (id === "publishers:alice") {
|
||||
return newPublisher;
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "skillSlugAliases") {
|
||||
return {
|
||||
withIndex: (indexName: string) => {
|
||||
expect(indexName).toBe("by_skill");
|
||||
return {
|
||||
collect: async () => aliases,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (indexName: string) => {
|
||||
expect(indexName).toBe("by_handle");
|
||||
return {
|
||||
unique: async () => newPublisher,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: (indexName: string) => {
|
||||
expect(indexName).toBe("by_publisher_user");
|
||||
return {
|
||||
unique: async () => existingMember,
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
patch,
|
||||
insert,
|
||||
},
|
||||
} as never,
|
||||
{
|
||||
actorUserId: "users:2",
|
||||
transferId: "skillOwnershipTransfers:1",
|
||||
} as never,
|
||||
)) as { ok: boolean; skillSlug: string };
|
||||
|
||||
expect(result).toEqual({ ok: true, skillSlug: "demo" });
|
||||
expect(patch).toHaveBeenCalledWith(
|
||||
"skills:1",
|
||||
expect.objectContaining({
|
||||
ownerUserId: "users:2",
|
||||
ownerPublisherId: "publishers:alice",
|
||||
}),
|
||||
);
|
||||
expect(patch).toHaveBeenCalledWith(
|
||||
"skillSlugAliases:1",
|
||||
expect.objectContaining({
|
||||
ownerUserId: "users:2",
|
||||
ownerPublisherId: "publishers:alice",
|
||||
}),
|
||||
);
|
||||
expect(patch).toHaveBeenCalledWith(
|
||||
"skillSlugAliases:2",
|
||||
expect.objectContaining({
|
||||
ownerUserId: "users:2",
|
||||
ownerPublisherId: "publishers:alice",
|
||||
}),
|
||||
);
|
||||
expect(patch).toHaveBeenCalledWith(
|
||||
"skillOwnershipTransfers:1",
|
||||
expect.objectContaining({ status: "accepted" }),
|
||||
);
|
||||
});
|
||||
|
||||
it("acceptTransferInternal cancels stale transfer when ownership changed", async () => {
|
||||
const patch = vi.fn(async () => {});
|
||||
|
||||
|
||||
@@ -1,6 +1,10 @@
|
||||
import { v } from "convex/values";
|
||||
import type { Doc, Id } from "./_generated/dataModel";
|
||||
import { internalMutation, internalQuery } from "./functions";
|
||||
import {
|
||||
ensurePersonalPublisherForUser,
|
||||
getActiveUserByHandleOrPersonalPublisher,
|
||||
} from "./lib/publishers";
|
||||
const TRANSFER_EXPIRY_MS = 7 * 24 * 60 * 60 * 1000;
|
||||
|
||||
type TransferDoc = Doc<"skillOwnershipTransfers">;
|
||||
@@ -111,11 +115,8 @@ export const requestTransferInternal = internalMutation({
|
||||
const toHandle = normalizeHandle(args.toUserHandle);
|
||||
if (!toHandle) throw new Error("toUserHandle required");
|
||||
|
||||
const toUser = await ctx.db
|
||||
.query("users")
|
||||
.withIndex("handle", (q) => q.eq("handle", toHandle))
|
||||
.first();
|
||||
if (!toUser || toUser.deletedAt || toUser.deactivatedAt) throw new Error("User not found");
|
||||
const toUser = await getActiveUserByHandleOrPersonalPublisher(ctx, toHandle);
|
||||
if (!toUser) throw new Error("User not found");
|
||||
if (toUser._id === args.actorUserId) throw new Error("Cannot transfer to yourself");
|
||||
|
||||
const activePending = await getActivePendingTransferForSkill(ctx, args.skillId, now);
|
||||
@@ -157,7 +158,7 @@ export const acceptTransferInternal = internalMutation({
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const now = Date.now();
|
||||
await requireActiveUserById(ctx, args.actorUserId);
|
||||
const newOwner = await requireActiveUserById(ctx, args.actorUserId);
|
||||
|
||||
const transfer = await validatePendingTransferForActor(ctx, {
|
||||
transferId: args.transferId,
|
||||
@@ -173,10 +174,27 @@ export const acceptTransferInternal = internalMutation({
|
||||
throw new Error("Transfer is no longer valid");
|
||||
}
|
||||
|
||||
const newPublisher = await ensurePersonalPublisherForUser(ctx, newOwner);
|
||||
if (!newPublisher) throw new Error("Failed to resolve publisher for new owner");
|
||||
|
||||
await ctx.db.patch(skill._id, {
|
||||
ownerUserId: args.actorUserId,
|
||||
ownerPublisherId: newPublisher._id,
|
||||
updatedAt: now,
|
||||
});
|
||||
|
||||
const aliases = await ctx.db
|
||||
.query("skillSlugAliases")
|
||||
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
|
||||
.collect();
|
||||
for (const alias of aliases) {
|
||||
await ctx.db.patch(alias._id, {
|
||||
ownerUserId: args.actorUserId,
|
||||
ownerPublisherId: newPublisher._id,
|
||||
updatedAt: now,
|
||||
});
|
||||
}
|
||||
|
||||
await ctx.db.patch(transfer._id, { status: "accepted", respondedAt: now });
|
||||
|
||||
await ctx.db.insert("auditLogs", {
|
||||
|
||||
@@ -0,0 +1,173 @@
|
||||
import { beforeEach, describe, expect, it, vi } from "vitest";
|
||||
|
||||
vi.mock("@convex-dev/auth/server", () => ({
|
||||
getAuthUserId: vi.fn(),
|
||||
authTables: {},
|
||||
}));
|
||||
|
||||
const { getAuthUserId } = await import("@convex-dev/auth/server");
|
||||
const { deleteTags } = await import("./skills");
|
||||
|
||||
type WrappedHandler<TArgs, TResult = unknown> = {
|
||||
_handler: (ctx: unknown, args: TArgs) => Promise<TResult>;
|
||||
};
|
||||
|
||||
const deleteTagsHandler = (
|
||||
deleteTags as unknown as WrappedHandler<{
|
||||
skillId: string;
|
||||
tags: string[];
|
||||
}>
|
||||
)._handler;
|
||||
|
||||
function buildGlobalStatsQuery(table: string) {
|
||||
if (table !== "globalStats") return null;
|
||||
return {
|
||||
withIndex: () => ({
|
||||
unique: async () => ({ _id: "globalStats:1", activeSkillsCount: 100 }),
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function buildDigestQuery(table: string) {
|
||||
if (table !== "skillSearchDigest") return null;
|
||||
return {
|
||||
withIndex: () => ({
|
||||
unique: async () => null,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
function makeCtx(params: { user: Record<string, unknown>; skill: Record<string, unknown> | null }) {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue(params.user._id as never);
|
||||
const patch = vi.fn(async (_id: string, value: Record<string, unknown>) => value);
|
||||
const db = {
|
||||
get: vi.fn(async (id: string) => {
|
||||
if (id === params.user._id) return params.user;
|
||||
if (params.skill && id === params.skill._id) return params.skill;
|
||||
return null;
|
||||
}),
|
||||
query: vi.fn((table: string) => {
|
||||
const globalStatsQuery = buildGlobalStatsQuery(table);
|
||||
if (globalStatsQuery) return globalStatsQuery;
|
||||
const digestQuery = buildDigestQuery(table);
|
||||
if (digestQuery) return digestQuery;
|
||||
throw new Error(`unexpected table ${table}`);
|
||||
}),
|
||||
insert: vi.fn(),
|
||||
patch,
|
||||
delete: vi.fn(),
|
||||
replace: vi.fn(),
|
||||
normalizeId: vi.fn(() => null),
|
||||
};
|
||||
const auth = { getUserIdentity: vi.fn(async () => ({ tokenIdentifier: "test" })) };
|
||||
return { db, auth, patch };
|
||||
}
|
||||
|
||||
const ownerUser = {
|
||||
_id: "users:owner",
|
||||
deletedAt: undefined,
|
||||
deactivatedAt: undefined,
|
||||
role: undefined,
|
||||
};
|
||||
|
||||
const modUser = {
|
||||
_id: "users:mod",
|
||||
deletedAt: undefined,
|
||||
deactivatedAt: undefined,
|
||||
role: "moderator",
|
||||
};
|
||||
|
||||
const otherUser = {
|
||||
_id: "users:other",
|
||||
deletedAt: undefined,
|
||||
deactivatedAt: undefined,
|
||||
role: undefined,
|
||||
};
|
||||
|
||||
const baseSkill = {
|
||||
_id: "skills:1",
|
||||
ownerUserId: "users:owner",
|
||||
tags: {
|
||||
latest: "versions:3",
|
||||
stable: "versions:2",
|
||||
beta: "versions:3",
|
||||
"old-tag": "versions:1",
|
||||
},
|
||||
moderationStatus: "active",
|
||||
moderationFlags: undefined,
|
||||
softDeletedAt: undefined,
|
||||
};
|
||||
|
||||
describe("deleteTags", () => {
|
||||
beforeEach(() => {
|
||||
vi.mocked(getAuthUserId).mockReset();
|
||||
});
|
||||
|
||||
it("deletes specified tags and keeps latest", async () => {
|
||||
const { db, auth, patch } = makeCtx({ user: ownerUser, skill: baseSkill });
|
||||
await deleteTagsHandler(
|
||||
{ db, auth } as never,
|
||||
{ skillId: "skills:1", tags: ["stable", "old-tag"] } as never,
|
||||
);
|
||||
expect(patch).toHaveBeenCalledOnce();
|
||||
const patchArgs = patch.mock.calls[0];
|
||||
expect(patchArgs[1]).toHaveProperty("tags");
|
||||
const newTags = (patchArgs[1] as Record<string, unknown>).tags as Record<string, string>;
|
||||
expect(newTags).toHaveProperty("latest");
|
||||
expect(newTags).toHaveProperty("beta");
|
||||
expect(newTags).not.toHaveProperty("stable");
|
||||
expect(newTags).not.toHaveProperty("old-tag");
|
||||
});
|
||||
|
||||
it("protects the latest tag from deletion", async () => {
|
||||
const { db, auth, patch } = makeCtx({ user: ownerUser, skill: baseSkill });
|
||||
await deleteTagsHandler(
|
||||
{ db, auth } as never,
|
||||
{ skillId: "skills:1", tags: ["latest"] } as never,
|
||||
);
|
||||
// No actual tag removed → no db.patch call
|
||||
expect(patch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("skips db write when no tags are actually removed", async () => {
|
||||
const { db, auth, patch } = makeCtx({ user: ownerUser, skill: baseSkill });
|
||||
await deleteTagsHandler(
|
||||
{ db, auth } as never,
|
||||
{ skillId: "skills:1", tags: ["nonexistent", "latest"] } as never,
|
||||
);
|
||||
expect(patch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("throws for non-owner non-moderator user", async () => {
|
||||
const { db, auth } = makeCtx({ user: otherUser, skill: baseSkill });
|
||||
await expect(
|
||||
deleteTagsHandler({ db, auth } as never, { skillId: "skills:1", tags: ["stable"] } as never),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
it("allows moderator to delete tags on other user's skill", async () => {
|
||||
const { db, auth, patch } = makeCtx({ user: modUser, skill: baseSkill });
|
||||
await deleteTagsHandler(
|
||||
{ db, auth } as never,
|
||||
{ skillId: "skills:1", tags: ["beta"] } as never,
|
||||
);
|
||||
expect(patch).toHaveBeenCalledOnce();
|
||||
const newTags = (patch.mock.calls[0][1] as Record<string, unknown>).tags as Record<
|
||||
string,
|
||||
string
|
||||
>;
|
||||
expect(newTags).not.toHaveProperty("beta");
|
||||
expect(newTags).toHaveProperty("latest");
|
||||
expect(newTags).toHaveProperty("stable");
|
||||
});
|
||||
|
||||
it("throws when skill not found", async () => {
|
||||
const { db, auth } = makeCtx({ user: ownerUser, skill: null });
|
||||
await expect(
|
||||
deleteTagsHandler(
|
||||
{ db, auth } as never,
|
||||
{ skillId: "skills:missing", tags: ["stable"] } as never,
|
||||
),
|
||||
).rejects.toThrow("Skill not found");
|
||||
});
|
||||
});
|
||||
@@ -33,6 +33,7 @@ const listPackageCatalogPageHandler = (
|
||||
family: "skill";
|
||||
channel: "official" | "community";
|
||||
isOfficial: boolean;
|
||||
capabilityTags: string[];
|
||||
}>;
|
||||
isDone: boolean;
|
||||
continueCursor: string;
|
||||
@@ -79,6 +80,7 @@ function makeDigest(
|
||||
changelog: "init",
|
||||
},
|
||||
tags: { latest: `skillVersions:${slug}-1` },
|
||||
capabilityTags: [],
|
||||
badges: {},
|
||||
stats: {
|
||||
downloads: 1,
|
||||
@@ -103,8 +105,13 @@ function makeDigest(
|
||||
};
|
||||
}
|
||||
|
||||
function makeCtx(pages: Array<{ page: Array<Record<string, unknown>>; isDone: boolean; continueCursor: string }>) {
|
||||
const pageByCursor = new Map<string | null, { page: Array<Record<string, unknown>>; isDone: boolean; continueCursor: string }>();
|
||||
function makeCtx(
|
||||
pages: Array<{ page: Array<Record<string, unknown>>; isDone: boolean; continueCursor: string }>,
|
||||
) {
|
||||
const pageByCursor = new Map<
|
||||
string | null,
|
||||
{ page: Array<Record<string, unknown>>; isDone: boolean; continueCursor: string }
|
||||
>();
|
||||
const allDigests = pages.flatMap((page) => page.page);
|
||||
let cursor: string | null = null;
|
||||
for (const page of pages) {
|
||||
@@ -116,7 +123,12 @@ function makeCtx(pages: Array<{ page: Array<Record<string, unknown>>; isDone: bo
|
||||
query: (table: string) => {
|
||||
if (table === "skills") {
|
||||
return {
|
||||
withIndex: (_index: string, builder: (q: { eq: (field: string, value: string) => { field: string; value: string } }) => { field: string; value: string }) => {
|
||||
withIndex: (
|
||||
_index: string,
|
||||
builder: (q: {
|
||||
eq: (field: string, value: string) => { field: string; value: string };
|
||||
}) => { field: string; value: string },
|
||||
) => {
|
||||
const constraint = builder({ eq: (field, value) => ({ field, value }) });
|
||||
return {
|
||||
unique: async () => {
|
||||
@@ -205,4 +217,48 @@ describe("skills package catalog queries", () => {
|
||||
});
|
||||
expect(result[0]?.score).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it("filters skills by capability tag", async () => {
|
||||
const result = await listPackageCatalogPageHandler(
|
||||
makeCtx([
|
||||
{
|
||||
page: [
|
||||
makeDigest("paytoll", { capabilityTags: ["crypto", "requires-wallet"] }),
|
||||
makeDigest("weather"),
|
||||
],
|
||||
isDone: true,
|
||||
continueCursor: "",
|
||||
},
|
||||
]),
|
||||
{
|
||||
capabilityTag: "crypto",
|
||||
paginationOpts: { cursor: null, numItems: 10 },
|
||||
},
|
||||
);
|
||||
|
||||
expect(result.page).toEqual([
|
||||
expect.objectContaining({
|
||||
name: "paytoll",
|
||||
capabilityTags: ["crypto", "requires-wallet"],
|
||||
}),
|
||||
]);
|
||||
});
|
||||
|
||||
it("returns empty immediately for unknown capability tags", async () => {
|
||||
const result = await listPackageCatalogPageHandler(
|
||||
makeCtx([
|
||||
{
|
||||
page: [makeDigest("paytoll", { capabilityTags: ["crypto", "requires-wallet"] })],
|
||||
isDone: true,
|
||||
continueCursor: "",
|
||||
},
|
||||
]),
|
||||
{
|
||||
capabilityTag: "not-a-real-tag",
|
||||
paginationOpts: { cursor: null, numItems: 10 },
|
||||
},
|
||||
);
|
||||
|
||||
expect(result).toEqual({ page: [], isDone: true, continueCursor: "" });
|
||||
});
|
||||
});
|
||||
|
||||
@@ -5,7 +5,10 @@ vi.mock("@convex-dev/auth/server", () => ({
|
||||
authTables: {},
|
||||
}));
|
||||
|
||||
import { getPendingScanSkillsInternal } from "./skills";
|
||||
import {
|
||||
getActiveSkillBatchForStaticScanBackfillInternal,
|
||||
getPendingScanSkillsInternal,
|
||||
} from "./skills";
|
||||
|
||||
type PendingScanResult = Array<{
|
||||
skillId: string;
|
||||
@@ -25,6 +28,17 @@ const getPendingScanSkillsHandler = (
|
||||
>
|
||||
)._handler;
|
||||
|
||||
const getStaticScanBackfillBatchHandler = (
|
||||
getActiveSkillBatchForStaticScanBackfillInternal as unknown as WrappedHandler<
|
||||
Record<string, unknown>,
|
||||
{
|
||||
skills: Array<{ skillId: string; versionId: string; slug: string }>;
|
||||
nextCursor: number;
|
||||
done: boolean;
|
||||
}
|
||||
>
|
||||
)._handler;
|
||||
|
||||
describe("skills.getPendingScanSkillsInternal", () => {
|
||||
it("includes unresolved VT records from the oldest slice and skips finalized ones", async () => {
|
||||
const recentSkills = [
|
||||
@@ -215,6 +229,114 @@ describe("skills.getPendingScanSkillsInternal", () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe("skills.getActiveSkillBatchForStaticScanBackfillInternal", () => {
|
||||
it("includes latest active skills with missing or stale static scan engine versions", async () => {
|
||||
const skills = [
|
||||
{
|
||||
_id: "skills:missing-static",
|
||||
_creationTime: 10,
|
||||
softDeletedAt: undefined,
|
||||
moderationStatus: "active",
|
||||
latestVersionId: "skillVersions:missing-static",
|
||||
slug: "missing-static",
|
||||
},
|
||||
{
|
||||
_id: "skills:stale-static",
|
||||
_creationTime: 20,
|
||||
softDeletedAt: undefined,
|
||||
moderationStatus: "active",
|
||||
latestVersionId: "skillVersions:stale-static",
|
||||
slug: "stale-static",
|
||||
},
|
||||
{
|
||||
_id: "skills:current-static",
|
||||
_creationTime: 30,
|
||||
softDeletedAt: undefined,
|
||||
moderationStatus: "active",
|
||||
latestVersionId: "skillVersions:current-static",
|
||||
slug: "current-static",
|
||||
},
|
||||
{
|
||||
_id: "skills:hidden-static",
|
||||
_creationTime: 40,
|
||||
softDeletedAt: undefined,
|
||||
moderationStatus: "hidden",
|
||||
latestVersionId: "skillVersions:hidden-static",
|
||||
slug: "hidden-static",
|
||||
},
|
||||
];
|
||||
|
||||
const versions = new Map<string, unknown>([
|
||||
["skillVersions:missing-static", { _id: "skillVersions:missing-static" }],
|
||||
[
|
||||
"skillVersions:stale-static",
|
||||
{
|
||||
_id: "skillVersions:stale-static",
|
||||
staticScan: { engineVersion: "v2.2.0" },
|
||||
},
|
||||
],
|
||||
[
|
||||
"skillVersions:current-static",
|
||||
{
|
||||
_id: "skillVersions:current-static",
|
||||
staticScan: { engineVersion: "v2.4.0" },
|
||||
},
|
||||
],
|
||||
[
|
||||
"skillVersions:hidden-static",
|
||||
{
|
||||
_id: "skillVersions:hidden-static",
|
||||
staticScan: { engineVersion: "v2.2.0" },
|
||||
},
|
||||
],
|
||||
]);
|
||||
|
||||
const ctx = {
|
||||
db: {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table !== "skills") throw new Error(`unexpected table ${table}`);
|
||||
return {
|
||||
withIndex: (
|
||||
indexName: string,
|
||||
builder: (q: { gt: (field: string, value: unknown) => unknown }) => unknown,
|
||||
) => {
|
||||
builder({ gt: () => ({}) });
|
||||
if (indexName !== "by_creation_time") {
|
||||
throw new Error(`unexpected index ${indexName}`);
|
||||
}
|
||||
return {
|
||||
order: () => ({
|
||||
take: async () => skills,
|
||||
}),
|
||||
};
|
||||
},
|
||||
};
|
||||
}),
|
||||
get: vi.fn(async (id: string) => versions.get(id) ?? null),
|
||||
},
|
||||
};
|
||||
|
||||
const result = await getStaticScanBackfillBatchHandler(ctx, {
|
||||
batchSize: 10,
|
||||
cursor: 0,
|
||||
});
|
||||
|
||||
expect(result.skills).toEqual([
|
||||
{
|
||||
skillId: "skills:missing-static",
|
||||
versionId: "skillVersions:missing-static",
|
||||
slug: "missing-static",
|
||||
},
|
||||
{
|
||||
skillId: "skills:stale-static",
|
||||
versionId: "skillVersions:stale-static",
|
||||
slug: "stale-static",
|
||||
},
|
||||
]);
|
||||
expect(result.done).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
function makeSkill(
|
||||
id: string,
|
||||
versionId: string,
|
||||
|
||||
@@ -35,6 +35,12 @@ const getBySlugHandler = (
|
||||
image: string | null;
|
||||
bio?: string | null;
|
||||
} | null;
|
||||
latestVersion?: {
|
||||
files?: Array<{
|
||||
path: string;
|
||||
contentType?: string;
|
||||
}>;
|
||||
} | null;
|
||||
} | null
|
||||
>
|
||||
)._handler;
|
||||
@@ -171,4 +177,70 @@ describe("skills.getBySlug", () => {
|
||||
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
|
||||
it("normalizes misleading file MIME types in public version metadata", async () => {
|
||||
const ctx = makeCtx({
|
||||
skill: {
|
||||
_id: "skills:1",
|
||||
_creationTime: 1,
|
||||
slug: "demo",
|
||||
displayName: "Demo",
|
||||
summary: "Public demo skill",
|
||||
ownerUserId: "users:1",
|
||||
canonicalSkillId: undefined,
|
||||
forkOf: undefined,
|
||||
latestVersionId: "skillVersions:1",
|
||||
tags: {},
|
||||
stats: {
|
||||
downloads: 10,
|
||||
installsCurrent: 2,
|
||||
installsAllTime: 5,
|
||||
stars: 3,
|
||||
versions: 1,
|
||||
comments: 0,
|
||||
},
|
||||
createdAt: 1,
|
||||
updatedAt: 2,
|
||||
moderationStatus: "active",
|
||||
moderationFlags: undefined,
|
||||
softDeletedAt: undefined,
|
||||
},
|
||||
owner: {
|
||||
_id: "users:1",
|
||||
_creationTime: 1,
|
||||
handle: "demo-owner",
|
||||
name: "Demo Owner",
|
||||
displayName: "Demo Owner",
|
||||
image: null,
|
||||
},
|
||||
latestVersion: {
|
||||
_id: "skillVersions:1",
|
||||
_creationTime: 2,
|
||||
skillId: "skills:1",
|
||||
version: "1.0.0",
|
||||
fingerprint: "abc",
|
||||
changelog: "",
|
||||
changelogSource: "user",
|
||||
files: [
|
||||
{
|
||||
path: "src/index.ts",
|
||||
size: 10,
|
||||
sha256: "deadbeef",
|
||||
contentType: "video/mp2t",
|
||||
},
|
||||
],
|
||||
createdBy: "users:1",
|
||||
createdAt: 2,
|
||||
},
|
||||
});
|
||||
|
||||
const result = await getBySlugHandler(ctx, { slug: "demo" } as never);
|
||||
|
||||
expect(result?.latestVersion?.files).toEqual([
|
||||
expect.objectContaining({
|
||||
path: "src/index.ts",
|
||||
contentType: "application/typescript",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
@@ -37,6 +37,7 @@ function makeCtx() {
|
||||
slug: "padel",
|
||||
displayName: "Padel",
|
||||
ownerUserId: "users:owner",
|
||||
ownerPublisherId: "publishers:local",
|
||||
latestVersionId: "skillVersions:1",
|
||||
manualOverride: {
|
||||
verdict: "clean",
|
||||
@@ -103,6 +104,15 @@ function makeCtx() {
|
||||
switch (id) {
|
||||
case "skillVersions:1":
|
||||
return latestVersion;
|
||||
case "publishers:local":
|
||||
return {
|
||||
_id: "publishers:local",
|
||||
_creationTime: 1,
|
||||
kind: "user",
|
||||
handle: "local-publisher",
|
||||
displayName: "Local Dev",
|
||||
linkedUserId: "users:owner",
|
||||
};
|
||||
case "users:owner":
|
||||
return {
|
||||
_id: "users:owner",
|
||||
@@ -150,7 +160,7 @@ describe("getBySlugForStaff audit logs", () => {
|
||||
vi.mocked(requireUser).mockReset();
|
||||
});
|
||||
|
||||
it("returns reviewer info and recent audit logs with actor handles", async () => {
|
||||
it("returns publisher-backed owner info plus recent audit logs with actor handles", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:moderator",
|
||||
user: { _id: "users:moderator", role: "moderator" },
|
||||
@@ -162,6 +172,7 @@ describe("getBySlugForStaff audit logs", () => {
|
||||
slug: "padel",
|
||||
auditLogLimit: 5,
|
||||
})) as {
|
||||
owner: { handle?: string | null } | null;
|
||||
overrideReviewer: { handle?: string | null } | null;
|
||||
auditLogs: Array<{
|
||||
actor: { handle?: string | null } | null;
|
||||
@@ -171,6 +182,7 @@ describe("getBySlugForStaff audit logs", () => {
|
||||
|
||||
expect(getSkillBadgeMap).toHaveBeenCalled();
|
||||
expect(auditTake).toHaveBeenCalledWith(5);
|
||||
expect(result.owner?.handle).toBe("local-publisher");
|
||||
expect(result.overrideReviewer?.handle).toBe("moddy");
|
||||
expect(result.auditLogs).toHaveLength(2);
|
||||
expect(result.auditLogs[0]?.action).toBe("skill.manual_override.set");
|
||||
|
||||
+518
-74
@@ -1,4 +1,5 @@
|
||||
import { getAuthUserId } from "@convex-dev/auth/server";
|
||||
import { normalizeTextContentType } from "clawhub-schema";
|
||||
import { getPage, type IndexKey, paginator } from "convex-helpers/server/pagination";
|
||||
import { paginationOptsValidator } from "convex/server";
|
||||
import { ConvexError, v, type Value } from "convex/values";
|
||||
@@ -38,6 +39,7 @@ import { deriveModerationFlags } from "./lib/moderation";
|
||||
import { buildModerationSnapshot } from "./lib/moderationEngine";
|
||||
import {
|
||||
legacyFlagsFromVerdict,
|
||||
MODERATION_ENGINE_VERSION,
|
||||
summarizeReasonCodes,
|
||||
verdictFromCodes,
|
||||
} from "./lib/moderationReasonCodes";
|
||||
@@ -66,15 +68,23 @@ import {
|
||||
reserveSlugForHardDeleteFinalize,
|
||||
upsertReservedSlugForRightfulOwner,
|
||||
} from "./lib/reservedSlugs";
|
||||
import { SKILL_CAPABILITY_TAGS } from "./lib/skillCapabilityTags";
|
||||
import {
|
||||
fetchText,
|
||||
type PublishResult,
|
||||
publishVersionForUser,
|
||||
queueHighlightedWebhook,
|
||||
} from "./lib/skillPublish";
|
||||
import { runStaticPublishScan } from "./lib/staticPublishScan";
|
||||
import { getFrontmatterValue, hashSkillFiles } from "./lib/skills";
|
||||
import { computeIsSuspicious, isSkillSuspicious } from "./lib/skillSafety";
|
||||
import { digestToHydratableSkill, digestToOwnerInfo } from "./lib/skillSearchDigest";
|
||||
import {
|
||||
digestToHydratableSkill,
|
||||
digestToOwnerInfo,
|
||||
extractDigestFields,
|
||||
upsertSkillSearchDigest,
|
||||
} from "./lib/skillSearchDigest";
|
||||
import { adjustUserSkillStatsForSkillChange } from "./lib/userSkillStats";
|
||||
import schema from "./schema";
|
||||
|
||||
export { publishVersionForUser } from "./lib/skillPublish";
|
||||
@@ -111,6 +121,7 @@ const DEFAULT_STAFF_AUDIT_LOG_LIMIT = 10;
|
||||
const MAX_STAFF_AUDIT_LOG_LIMIT = 50;
|
||||
const USER_MODERATION_REASON = "user.moderation";
|
||||
const SKILL_CATALOG_CURSOR_PREFIX = "skillcat:";
|
||||
const SKILL_CAPABILITY_TAG_SET = new Set<string>(SKILL_CAPABILITY_TAGS);
|
||||
|
||||
function buildStructuredModerationPatch(params: {
|
||||
staticScan?: Doc<"skillVersions">["staticScan"];
|
||||
@@ -316,6 +327,8 @@ const NONSUSPICIOUS_SORT_INDEXES = {
|
||||
stars: "by_nonsuspicious_stars",
|
||||
installs: "by_nonsuspicious_installs",
|
||||
} as const;
|
||||
const MAX_FILTERED_PUBLIC_LIST_SCAN_PAGES = 12;
|
||||
const MAX_FILTERED_PUBLIC_LIST_SCAN_ROWS = 500;
|
||||
|
||||
function isSkillVersionId(
|
||||
value: Id<"skillVersions"> | null | undefined,
|
||||
@@ -450,7 +463,7 @@ async function syncSkillModerationFromLatestVersion(
|
||||
|
||||
function buildConflictingSkillUrl(
|
||||
skill: Doc<"skills">,
|
||||
owner: Doc<"users"> | Doc<"publishers"> | null | undefined,
|
||||
owner: SkillOwnerRef,
|
||||
) {
|
||||
if (!owner || owner.deletedAt || owner.deactivatedAt || !isPublicSkillDoc(skill)) return null;
|
||||
const ownerParam = owner.handle?.trim() || String(owner._id);
|
||||
@@ -460,7 +473,7 @@ function buildConflictingSkillUrl(
|
||||
|
||||
function buildSlugTakenErrorMessage(
|
||||
skill: Doc<"skills">,
|
||||
owner: Doc<"users"> | Doc<"publishers"> | null | undefined,
|
||||
owner: SkillOwnerRef,
|
||||
) {
|
||||
if (!owner || owner.deletedAt || owner.deactivatedAt) {
|
||||
return (
|
||||
@@ -476,7 +489,7 @@ function buildSlugTakenErrorMessage(
|
||||
|
||||
function buildAliasTakenErrorMessage(
|
||||
skill: Doc<"skills">,
|
||||
owner: Doc<"users"> | Doc<"publishers"> | null | undefined,
|
||||
owner: SkillOwnerRef,
|
||||
) {
|
||||
const base = "Slug redirects to an existing skill. Choose a different slug.";
|
||||
const url = buildConflictingSkillUrl(skill, owner);
|
||||
@@ -488,6 +501,16 @@ function normalizeSkillSlugKey(slug: string) {
|
||||
return slug.trim().toLowerCase();
|
||||
}
|
||||
|
||||
type SkillOwnerRef =
|
||||
| {
|
||||
_id: Id<"users"> | Id<"publishers">;
|
||||
handle?: string | null;
|
||||
deletedAt?: number | null;
|
||||
deactivatedAt?: number | null;
|
||||
}
|
||||
| null
|
||||
| undefined;
|
||||
|
||||
function normalizeSkillSlugForWrite(slug: string) {
|
||||
const normalized = normalizeSkillSlugKey(slug);
|
||||
if (!normalized || !/^[a-z0-9][a-z0-9-]*$/.test(normalized)) {
|
||||
@@ -736,6 +759,7 @@ async function hardDeleteSkillStep(
|
||||
const nextSkill = { ...skill, ...patch };
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, nextSkill);
|
||||
}
|
||||
|
||||
switch (phase) {
|
||||
@@ -1041,6 +1065,7 @@ type PublicSkillVersion = {
|
||||
createdBy?: Id<"users">;
|
||||
createdAt?: number;
|
||||
softDeletedAt?: number;
|
||||
capabilityTags?: string[];
|
||||
sha256hash?: string;
|
||||
vtAnalysis?: Doc<"skillVersions">["vtAnalysis"];
|
||||
llmAnalysis?: Doc<"skillVersions">["llmAnalysis"];
|
||||
@@ -1207,7 +1232,7 @@ function toPublicSkillVersion(
|
||||
path: file.path,
|
||||
size: file.size,
|
||||
sha256: file.sha256,
|
||||
contentType: file.contentType,
|
||||
contentType: normalizeTextContentType(file.path, file.contentType),
|
||||
})),
|
||||
parsed: version.parsed
|
||||
? {
|
||||
@@ -1218,6 +1243,7 @@ function toPublicSkillVersion(
|
||||
createdBy: version.createdBy,
|
||||
createdAt: version.createdAt,
|
||||
softDeletedAt: version.softDeletedAt,
|
||||
capabilityTags: version.capabilityTags,
|
||||
sha256hash: version.sha256hash,
|
||||
vtAnalysis: version.vtAnalysis,
|
||||
llmAnalysis: version.llmAnalysis,
|
||||
@@ -1632,7 +1658,11 @@ export const getBySlugForStaff = query({
|
||||
if (!skill) return null;
|
||||
|
||||
const latestVersion = skill.latestVersionId ? await ctx.db.get(skill.latestVersionId) : null;
|
||||
const owner = toPublicUser(await ctx.db.get(skill.ownerUserId));
|
||||
const ownerPublisher = await getOwnerPublisher(ctx, {
|
||||
ownerPublisherId: skill.ownerPublisherId,
|
||||
ownerUserId: skill.ownerUserId,
|
||||
});
|
||||
const owner = toPublicPublisher(ownerPublisher);
|
||||
const badges = await getSkillBadgeMap(ctx, skill._id);
|
||||
const rawAuditLogs = await ctx.db
|
||||
.query("auditLogs")
|
||||
@@ -1659,10 +1689,20 @@ export const getBySlugForStaff = query({
|
||||
}));
|
||||
|
||||
const forkOfSkill = skill.forkOf?.skillId ? await ctx.db.get(skill.forkOf.skillId) : null;
|
||||
const forkOfOwner = forkOfSkill ? await ctx.db.get(forkOfSkill.ownerUserId) : null;
|
||||
const forkOfOwner = forkOfSkill
|
||||
? await getOwnerPublisher(ctx, {
|
||||
ownerPublisherId: forkOfSkill.ownerPublisherId,
|
||||
ownerUserId: forkOfSkill.ownerUserId,
|
||||
})
|
||||
: null;
|
||||
|
||||
const canonicalSkill = skill.canonicalSkillId ? await ctx.db.get(skill.canonicalSkillId) : null;
|
||||
const canonicalOwner = canonicalSkill ? await ctx.db.get(canonicalSkill.ownerUserId) : null;
|
||||
const canonicalOwner = canonicalSkill
|
||||
? await getOwnerPublisher(ctx, {
|
||||
ownerPublisherId: canonicalSkill.ownerPublisherId,
|
||||
ownerUserId: canonicalSkill.ownerUserId,
|
||||
})
|
||||
: null;
|
||||
|
||||
return {
|
||||
requestedSlug: resolved.requestedSlug,
|
||||
@@ -1681,8 +1721,8 @@ export const getBySlugForStaff = query({
|
||||
displayName: forkOfSkill.displayName,
|
||||
},
|
||||
owner: {
|
||||
handle: forkOfOwner?.handle ?? forkOfOwner?.name ?? null,
|
||||
userId: forkOfOwner?._id ?? null,
|
||||
handle: forkOfOwner?.handle ?? null,
|
||||
userId: forkOfOwner?.linkedUserId ?? null,
|
||||
},
|
||||
}
|
||||
: null,
|
||||
@@ -1693,8 +1733,8 @@ export const getBySlugForStaff = query({
|
||||
displayName: canonicalSkill.displayName,
|
||||
},
|
||||
owner: {
|
||||
handle: canonicalOwner?.handle ?? canonicalOwner?.name ?? null,
|
||||
userId: canonicalOwner?._id ?? null,
|
||||
handle: canonicalOwner?.handle ?? null,
|
||||
userId: canonicalOwner?.linkedUserId ?? null,
|
||||
},
|
||||
}
|
||||
: null,
|
||||
@@ -2055,7 +2095,8 @@ export const list = query({
|
||||
)
|
||||
.unique());
|
||||
const isOwnDashboard = Boolean(
|
||||
membership || (userId && ownerPublisher?.kind === "user" && ownerPublisher.linkedUserId === userId),
|
||||
membership ||
|
||||
(userId && ownerPublisher?.kind === "user" && ownerPublisher.linkedUserId === userId),
|
||||
);
|
||||
const scopedEntries = await ctx.db
|
||||
.query("skills")
|
||||
@@ -2490,6 +2531,7 @@ export const report = mutation({
|
||||
const nextSkill = { ...skill, ...updates };
|
||||
await ctx.db.patch(skill._id, updates);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, nextSkill);
|
||||
|
||||
if (shouldAutoHide) {
|
||||
await setSkillEmbeddingsSoftDeleted(ctx, skill._id, true, now);
|
||||
@@ -2670,8 +2712,12 @@ export const listPublicPageV4 = query({
|
||||
dir: v.optional(v.union(v.literal("asc"), v.literal("desc"))),
|
||||
highlightedOnly: v.optional(v.boolean()),
|
||||
nonSuspiciousOnly: v.optional(v.boolean()),
|
||||
capabilityTag: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
if (args.capabilityTag && !isKnownSkillCapabilityTag(args.capabilityTag)) {
|
||||
return { page: [], hasMore: false, nextCursor: null };
|
||||
}
|
||||
const sort = args.sort ?? "newest";
|
||||
const dir = args.dir ?? (sort === "name" ? "asc" : "desc");
|
||||
const numItems = clampInt(args.numItems ?? 25, 1, MAX_PUBLIC_LIST_LIMIT);
|
||||
@@ -2684,6 +2730,7 @@ export const listPublicPageV4 = query({
|
||||
sort,
|
||||
dir,
|
||||
numItems,
|
||||
capabilityTag: args.capabilityTag,
|
||||
nonSuspiciousOnly: args.nonSuspiciousOnly ?? false,
|
||||
});
|
||||
}
|
||||
@@ -2700,46 +2747,113 @@ export const listPublicPageV4 = query({
|
||||
const isFirstPage = !decodedCursor;
|
||||
const startIndexKey: IndexKey = decodedCursor ?? eqPrefix;
|
||||
|
||||
const result = await getPage(ctx, {
|
||||
table: "skillSearchDigest",
|
||||
startIndexKey,
|
||||
startInclusive: isFirstPage,
|
||||
endIndexKey: eqPrefix,
|
||||
endInclusive: true,
|
||||
absoluteMaxRows: numItems,
|
||||
order: dir,
|
||||
index: indexName,
|
||||
schema,
|
||||
});
|
||||
|
||||
// Build PublicSkillEntry[] from digests
|
||||
const items: PublicSkillEntry[] = [];
|
||||
for (const digest of result.page) {
|
||||
const hydratable = digestToHydratableSkill(digest);
|
||||
const publicSkill = toPublicSkill(hydratable);
|
||||
if (!publicSkill) continue;
|
||||
const ownerInfo = digestToOwnerInfo(digest);
|
||||
if (!ownerInfo?.owner) continue;
|
||||
const latestVersion = digest.latestVersionSummary
|
||||
? toPublicSkillListVersionFromSummary(digest.latestVersionSummary, digest.latestVersionId)
|
||||
: null;
|
||||
items.push({
|
||||
skill: publicSkill,
|
||||
latestVersion,
|
||||
ownerHandle: ownerInfo.ownerHandle,
|
||||
owner: ownerInfo.owner,
|
||||
if (!args.capabilityTag) {
|
||||
const result = await getPage(ctx, {
|
||||
table: "skillSearchDigest",
|
||||
startIndexKey,
|
||||
startInclusive: isFirstPage,
|
||||
endIndexKey: eqPrefix,
|
||||
endInclusive: true,
|
||||
absoluteMaxRows: numItems,
|
||||
order: dir,
|
||||
index: indexName,
|
||||
schema,
|
||||
});
|
||||
|
||||
const items = result.page
|
||||
.map((digest) => buildPublicSkillEntryFromDigest(digest))
|
||||
.filter((item): item is PublicSkillEntry => item !== null);
|
||||
let nextCursor: string | null = null;
|
||||
if (result.hasMore && result.indexKeys.length > 0) {
|
||||
nextCursor = encodeIndexKey(result.indexKeys[result.indexKeys.length - 1]);
|
||||
}
|
||||
|
||||
return { page: items, hasMore: result.hasMore, nextCursor };
|
||||
}
|
||||
|
||||
const items: PublicSkillEntry[] = [];
|
||||
let scanCursor = startIndexKey;
|
||||
let scanInclusive = isFirstPage;
|
||||
let hasMore = false;
|
||||
let nextCursor: string | null = null;
|
||||
if (result.hasMore && result.indexKeys.length > 0) {
|
||||
nextCursor = encodeIndexKey(result.indexKeys[result.indexKeys.length - 1]);
|
||||
let remainingRows = Math.max(numItems, Math.min(MAX_FILTERED_PUBLIC_LIST_SCAN_ROWS, numItems * 12));
|
||||
|
||||
for (let pageCount = 0; pageCount < MAX_FILTERED_PUBLIC_LIST_SCAN_PAGES; pageCount += 1) {
|
||||
if (remainingRows <= 0) break;
|
||||
const batchSize = Math.min(remainingRows, Math.max(numItems * 3, numItems));
|
||||
const result = await getPage(ctx, {
|
||||
table: "skillSearchDigest",
|
||||
startIndexKey: scanCursor,
|
||||
startInclusive: scanInclusive,
|
||||
endIndexKey: eqPrefix,
|
||||
endInclusive: true,
|
||||
absoluteMaxRows: batchSize,
|
||||
order: dir,
|
||||
index: indexName,
|
||||
schema,
|
||||
});
|
||||
remainingRows -= batchSize;
|
||||
if (result.indexKeys.length === 0) {
|
||||
hasMore = false;
|
||||
nextCursor = null;
|
||||
break;
|
||||
}
|
||||
|
||||
for (let index = 0; index < result.page.length; index += 1) {
|
||||
const digest = result.page[index];
|
||||
const cursor = result.indexKeys[index];
|
||||
if ((digest.capabilityTags ?? []).includes(args.capabilityTag)) {
|
||||
const item = buildPublicSkillEntryFromDigest(digest);
|
||||
if (item) items.push(item);
|
||||
}
|
||||
if (items.length >= numItems) {
|
||||
hasMore = result.hasMore || index < result.page.length - 1;
|
||||
nextCursor = hasMore ? encodeIndexKey(cursor) : null;
|
||||
return { page: items, hasMore, nextCursor };
|
||||
}
|
||||
}
|
||||
|
||||
if (!result.hasMore) {
|
||||
hasMore = false;
|
||||
nextCursor = null;
|
||||
break;
|
||||
}
|
||||
|
||||
scanCursor = result.indexKeys[result.indexKeys.length - 1];
|
||||
scanInclusive = false;
|
||||
hasMore = true;
|
||||
nextCursor = encodeIndexKey(scanCursor);
|
||||
}
|
||||
|
||||
return { page: items, hasMore: result.hasMore, nextCursor };
|
||||
// Guard: never signal more pages when the scan budget is exhausted
|
||||
// without finding any items — that would cause the client's
|
||||
// IntersectionObserver auto-load to loop on empty responses.
|
||||
if (items.length === 0) {
|
||||
hasMore = false;
|
||||
nextCursor = null;
|
||||
}
|
||||
|
||||
return { page: items, hasMore, nextCursor };
|
||||
},
|
||||
});
|
||||
|
||||
function buildPublicSkillEntryFromDigest(digest: Doc<"skillSearchDigest">): PublicSkillEntry | null {
|
||||
const hydratable = digestToHydratableSkill(digest);
|
||||
const publicSkill = toPublicSkill(hydratable);
|
||||
if (!publicSkill) return null;
|
||||
const ownerInfo = digestToOwnerInfo(digest);
|
||||
if (!ownerInfo?.owner) return null;
|
||||
const latestVersion = digest.latestVersionSummary
|
||||
? toPublicSkillListVersionFromSummary(digest.latestVersionSummary, digest.latestVersionId)
|
||||
: null;
|
||||
return {
|
||||
skill: publicSkill,
|
||||
latestVersion,
|
||||
ownerHandle: ownerInfo.ownerHandle,
|
||||
owner: ownerInfo.owner,
|
||||
};
|
||||
}
|
||||
|
||||
type PublicSkillCatalogItem = {
|
||||
name: string;
|
||||
displayName: string;
|
||||
@@ -2775,12 +2889,13 @@ function decodeSkillCatalogCursor(raw: string | null | undefined): SkillCatalogC
|
||||
return { cursor: raw, offset: 0, pageSize: null, done: false };
|
||||
}
|
||||
try {
|
||||
const parsed = JSON.parse(raw.slice(SKILL_CATALOG_CURSOR_PREFIX.length)) as Partial<SkillCatalogCursorState>;
|
||||
const parsed = JSON.parse(
|
||||
raw.slice(SKILL_CATALOG_CURSOR_PREFIX.length),
|
||||
) as Partial<SkillCatalogCursorState>;
|
||||
return {
|
||||
cursor: typeof parsed.cursor === "string" ? parsed.cursor : null,
|
||||
offset: typeof parsed.offset === "number" && parsed.offset > 0 ? parsed.offset : 0,
|
||||
pageSize:
|
||||
typeof parsed.pageSize === "number" && parsed.pageSize > 0 ? parsed.pageSize : null,
|
||||
pageSize: typeof parsed.pageSize === "number" && parsed.pageSize > 0 ? parsed.pageSize : null,
|
||||
done: parsed.done === true,
|
||||
};
|
||||
} catch {
|
||||
@@ -2814,12 +2929,13 @@ function skillCatalogMatchesFilters(
|
||||
) {
|
||||
if (!isVisibleSkillCatalogDigest(digest)) return false;
|
||||
if (args.channel === "private") return false;
|
||||
if (args.capabilityTag) return false;
|
||||
if (args.executesCode === true) return false;
|
||||
const isOfficial = isSkillCatalogOfficial(digest);
|
||||
const channel = getSkillCatalogChannel(digest);
|
||||
if (typeof args.isOfficial === "boolean" && isOfficial !== args.isOfficial) return false;
|
||||
if (args.channel && channel !== args.channel) return false;
|
||||
if (args.capabilityTag && !(digest.capabilityTags ?? []).includes(args.capabilityTag))
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -2837,7 +2953,7 @@ function toPublicSkillCatalogItem(digest: Doc<"skillSearchDigest">): PublicSkill
|
||||
createdAt: digest.createdAt,
|
||||
updatedAt: digest.updatedAt,
|
||||
latestVersion: digest.latestVersionSummary?.version ?? null,
|
||||
capabilityTags: [],
|
||||
capabilityTags: digest.capabilityTags ?? [],
|
||||
executesCode: false,
|
||||
verificationTier: null,
|
||||
};
|
||||
@@ -2862,16 +2978,25 @@ function scoreSkillCatalogResult(digest: Doc<"skillSearchDigest">, queryText: st
|
||||
return score;
|
||||
}
|
||||
|
||||
function isKnownSkillCapabilityTag(tag: string | undefined) {
|
||||
return typeof tag === "string" && SKILL_CAPABILITY_TAG_SET.has(tag);
|
||||
}
|
||||
|
||||
export const listPackageCatalogPage = query({
|
||||
args: {
|
||||
channel: v.optional(v.union(v.literal("official"), v.literal("community"), v.literal("private"))),
|
||||
channel: v.optional(
|
||||
v.union(v.literal("official"), v.literal("community"), v.literal("private")),
|
||||
),
|
||||
isOfficial: v.optional(v.boolean()),
|
||||
executesCode: v.optional(v.boolean()),
|
||||
capabilityTag: v.optional(v.string()),
|
||||
paginationOpts: paginationOptsValidator,
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
if (args.channel === "private" || args.executesCode === true || args.capabilityTag) {
|
||||
if (args.capabilityTag && !isKnownSkillCapabilityTag(args.capabilityTag)) {
|
||||
return { page: [], isDone: true, continueCursor: "" };
|
||||
}
|
||||
if (args.channel === "private" || args.executesCode === true) {
|
||||
return { page: [], isDone: true, continueCursor: "" };
|
||||
}
|
||||
|
||||
@@ -2894,7 +3019,9 @@ export const listPackageCatalogPage = query({
|
||||
loops += 1;
|
||||
const effectivePageSize = Math.min(
|
||||
remainingScanBudget,
|
||||
offset > 0 && pageSize ? Math.max(pageSize, offset + 1) : Math.max(targetCount * 3, targetCount),
|
||||
offset > 0 && pageSize
|
||||
? Math.max(pageSize, offset + 1)
|
||||
: Math.max(targetCount * 3, targetCount),
|
||||
);
|
||||
if (effectivePageSize <= 0) break;
|
||||
remainingScanBudget -= effectivePageSize;
|
||||
@@ -2948,7 +3075,9 @@ export const searchPackageCatalogPublic = query({
|
||||
args: {
|
||||
query: v.string(),
|
||||
limit: v.optional(v.number()),
|
||||
channel: v.optional(v.union(v.literal("official"), v.literal("community"), v.literal("private"))),
|
||||
channel: v.optional(
|
||||
v.union(v.literal("official"), v.literal("community"), v.literal("private")),
|
||||
),
|
||||
isOfficial: v.optional(v.boolean()),
|
||||
executesCode: v.optional(v.boolean()),
|
||||
capabilityTag: v.optional(v.string()),
|
||||
@@ -2956,7 +3085,8 @@ export const searchPackageCatalogPublic = query({
|
||||
handler: async (ctx, args) => {
|
||||
const queryText = args.query.trim().toLowerCase();
|
||||
if (!queryText) return [];
|
||||
if (args.channel === "private" || args.executesCode === true || args.capabilityTag) return [];
|
||||
if (args.capabilityTag && !isKnownSkillCapabilityTag(args.capabilityTag)) return [];
|
||||
if (args.channel === "private" || args.executesCode === true) return [];
|
||||
|
||||
const targetCount = Math.max(1, Math.min(args.limit ?? 20, 100));
|
||||
const matches: Array<{ score: number; package: PublicSkillCatalogItem }> = [];
|
||||
@@ -3020,6 +3150,7 @@ async function fetchHighlightedPage(
|
||||
sort: SortKey;
|
||||
dir: "asc" | "desc";
|
||||
numItems: number;
|
||||
capabilityTag?: string;
|
||||
nonSuspiciousOnly: boolean;
|
||||
},
|
||||
) {
|
||||
@@ -3039,6 +3170,7 @@ async function fetchHighlightedPage(
|
||||
.unique();
|
||||
if (!digest || digest.softDeletedAt) continue;
|
||||
if (opts.nonSuspiciousOnly && digest.isSuspicious) continue;
|
||||
if (opts.capabilityTag && !(digest.capabilityTags ?? []).includes(opts.capabilityTag)) continue;
|
||||
digests.push(digest);
|
||||
}
|
||||
|
||||
@@ -3065,23 +3197,9 @@ async function fetchHighlightedPage(
|
||||
const trimmed = digests.slice(0, opts.numItems);
|
||||
|
||||
// Build PublicSkillEntry[]
|
||||
const items: PublicSkillEntry[] = [];
|
||||
for (const digest of trimmed) {
|
||||
const hydratable = digestToHydratableSkill(digest);
|
||||
const publicSkill = toPublicSkill(hydratable);
|
||||
if (!publicSkill) continue;
|
||||
const ownerInfo = digestToOwnerInfo(digest);
|
||||
if (!ownerInfo?.owner) continue;
|
||||
const latestVersion = digest.latestVersionSummary
|
||||
? toPublicSkillListVersionFromSummary(digest.latestVersionSummary, digest.latestVersionId)
|
||||
: null;
|
||||
items.push({
|
||||
skill: publicSkill,
|
||||
latestVersion,
|
||||
ownerHandle: ownerInfo.ownerHandle,
|
||||
owner: ownerInfo.owner,
|
||||
});
|
||||
}
|
||||
const items = trimmed
|
||||
.map((digest) => buildPublicSkillEntryFromDigest(digest))
|
||||
.filter((item): item is PublicSkillEntry => item !== null);
|
||||
|
||||
// Highlighted skills are few enough to return in one page — no cursor needed
|
||||
return { page: items, hasMore: false, nextCursor: null };
|
||||
@@ -3581,6 +3699,56 @@ export const getActiveSkillBatchForLlmBackfillInternal = internalQuery({
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* Get active latest skill versions whose static scan is missing or uses an older engine version.
|
||||
* Used to backfill new static rules onto already-published skills.
|
||||
*/
|
||||
export const getActiveSkillBatchForStaticScanBackfillInternal = internalQuery({
|
||||
args: {
|
||||
cursor: v.optional(v.number()),
|
||||
batchSize: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const batchSize = args.batchSize ?? 25;
|
||||
const cursor = args.cursor ?? 0;
|
||||
|
||||
const candidates = await ctx.db
|
||||
.query("skills")
|
||||
.withIndex("by_creation_time", (q) => q.gt("_creationTime", cursor))
|
||||
.order("asc")
|
||||
.take(batchSize * 4);
|
||||
|
||||
const results: Array<{
|
||||
skillId: Id<"skills">;
|
||||
versionId: Id<"skillVersions">;
|
||||
slug: string;
|
||||
}> = [];
|
||||
let nextCursor = cursor;
|
||||
|
||||
for (const skill of candidates) {
|
||||
nextCursor = skill._creationTime;
|
||||
if (results.length >= batchSize) break;
|
||||
|
||||
if (skill.softDeletedAt) continue;
|
||||
if ((skill.moderationStatus ?? "active") !== "active") continue;
|
||||
if (!skill.latestVersionId) continue;
|
||||
|
||||
const version = await ctx.db.get(skill.latestVersionId);
|
||||
if (!version) continue;
|
||||
if (version.staticScan?.engineVersion === MODERATION_ENGINE_VERSION) continue;
|
||||
|
||||
results.push({
|
||||
skillId: skill._id,
|
||||
versionId: version._id,
|
||||
slug: skill.slug,
|
||||
});
|
||||
}
|
||||
|
||||
const done = candidates.length < batchSize * 4;
|
||||
return { skills: results, nextCursor, done };
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* Get skills with stale moderationReason that have vtAnalysis cached.
|
||||
* Used to sync moderationReason with cached VT results.
|
||||
@@ -3679,6 +3847,159 @@ export const getPendingVTSkillsInternal = internalQuery({
|
||||
},
|
||||
});
|
||||
|
||||
export const updateSkillVersionStaticScanInternal = internalMutation({
|
||||
args: {
|
||||
skillId: v.id("skills"),
|
||||
versionId: v.id("skillVersions"),
|
||||
staticScan: v.object({
|
||||
status: v.union(v.literal("clean"), v.literal("suspicious"), v.literal("malicious")),
|
||||
reasonCodes: v.array(v.string()),
|
||||
findings: v.array(
|
||||
v.object({
|
||||
code: v.string(),
|
||||
severity: v.union(v.literal("info"), v.literal("warn"), v.literal("critical")),
|
||||
file: v.string(),
|
||||
line: v.number(),
|
||||
message: v.string(),
|
||||
evidence: v.string(),
|
||||
}),
|
||||
),
|
||||
summary: v.string(),
|
||||
engineVersion: v.string(),
|
||||
checkedAt: v.number(),
|
||||
}),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const version = await ctx.db.get(args.versionId);
|
||||
if (!version || version.skillId !== args.skillId) return { ok: true as const, skipped: "missing" as const };
|
||||
|
||||
await ctx.db.patch(version._id, {
|
||||
staticScan: args.staticScan,
|
||||
});
|
||||
|
||||
const skill = await ctx.db.get(args.skillId);
|
||||
if (!skill) return { ok: true as const, skipped: "missing" as const };
|
||||
if (skill.latestVersionId !== version._id) {
|
||||
return { ok: true as const, skipped: "not_latest" as const };
|
||||
}
|
||||
|
||||
const owner = skill.ownerUserId ? await ctx.db.get(skill.ownerUserId) : null;
|
||||
const now = Date.now();
|
||||
const updatedVersion = { ...version, staticScan: args.staticScan };
|
||||
const basePatch = buildScannerModerationPatchFromVersion({
|
||||
owner,
|
||||
version: updatedVersion,
|
||||
now,
|
||||
});
|
||||
const patch = applySkillManualOverrideToSkillPatch({
|
||||
skill,
|
||||
basePatch: {
|
||||
...basePatch,
|
||||
updatedAt: now,
|
||||
},
|
||||
now,
|
||||
});
|
||||
const nextSkill = { ...skill, ...patch };
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
|
||||
if (patch.moderationVerdict === "malicious" && skill.ownerUserId) {
|
||||
await ctx.scheduler.runAfter(0, internal.users.placeUserUnderModerationInternal, {
|
||||
ownerUserId: skill.ownerUserId,
|
||||
slug: skill.slug,
|
||||
reason:
|
||||
patch.moderationReasonCodes?.find((code) => code.startsWith("malicious.")) ??
|
||||
"malicious.static_scan",
|
||||
});
|
||||
}
|
||||
|
||||
return { ok: true as const, status: args.staticScan.status };
|
||||
},
|
||||
});
|
||||
|
||||
export const scanSkillVersionStaticallyInternal: ReturnType<typeof internalAction> = internalAction({
|
||||
args: {
|
||||
skillId: v.id("skills"),
|
||||
versionId: v.id("skillVersions"),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const [skill, version] = await Promise.all([
|
||||
ctx.runQuery(internal.skills.getSkillByIdInternal, { skillId: args.skillId }),
|
||||
ctx.runQuery(internal.skills.getVersionByIdInternal, { versionId: args.versionId }),
|
||||
]);
|
||||
|
||||
if (!skill || !version) {
|
||||
return { ok: true as const, skipped: "missing" as const };
|
||||
}
|
||||
|
||||
const staticScan = await runStaticPublishScan(ctx, {
|
||||
slug: skill.slug,
|
||||
displayName: skill.displayName,
|
||||
summary: skill.summary ?? undefined,
|
||||
frontmatter: version.parsed?.frontmatter ?? {},
|
||||
metadata: version.parsed?.metadata,
|
||||
files: version.files,
|
||||
});
|
||||
|
||||
return await ctx.runMutation(internal.skills.updateSkillVersionStaticScanInternal, {
|
||||
skillId: skill._id,
|
||||
versionId: version._id,
|
||||
staticScan,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const backfillSkillStaticScansInternal: ReturnType<typeof internalAction> = internalAction({
|
||||
args: {
|
||||
cursor: v.optional(v.number()),
|
||||
batchSize: v.optional(v.number()),
|
||||
rescanned: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const batchSize = Math.max(1, Math.min(args.batchSize ?? 25, 100));
|
||||
const batch = await ctx.runQuery(internal.skills.getActiveSkillBatchForStaticScanBackfillInternal, {
|
||||
cursor: args.cursor,
|
||||
batchSize,
|
||||
});
|
||||
|
||||
let rescanned = args.rescanned ?? 0;
|
||||
for (const skill of batch.skills) {
|
||||
await ctx.scheduler.runAfter(0, internal.skills.scanSkillVersionStaticallyInternal, {
|
||||
skillId: skill.skillId,
|
||||
versionId: skill.versionId,
|
||||
});
|
||||
rescanned += 1;
|
||||
}
|
||||
|
||||
if (!batch.done) {
|
||||
await ctx.scheduler.runAfter(0, internal.skills.backfillSkillStaticScansInternal, {
|
||||
cursor: batch.nextCursor,
|
||||
batchSize,
|
||||
rescanned,
|
||||
});
|
||||
}
|
||||
|
||||
return {
|
||||
rescanned,
|
||||
nextCursor: batch.nextCursor,
|
||||
done: batch.done,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const backfillSkillStaticScans: ReturnType<typeof action> = action({
|
||||
args: {
|
||||
batchSize: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const { user } = await requireUserFromAction(ctx);
|
||||
assertAdmin(user);
|
||||
return await ctx.runAction(internal.skills.backfillSkillStaticScansInternal, {
|
||||
batchSize: args.batchSize,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
* Emergency escalation by skillId for legacy rows without sha256hash.
|
||||
* Rebuilds the full moderation snapshot so legacy rows stay in sync with structured fields.
|
||||
@@ -3937,6 +4258,7 @@ export const applyBanToOwnedSkillsBatchInternal = internalMutation({
|
||||
const nextSkill = { ...skill, ...patch };
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, nextSkill);
|
||||
await setSkillEmbeddingsSoftDeleted(ctx, skill._id, true, args.bannedAt);
|
||||
}
|
||||
|
||||
@@ -4048,6 +4370,7 @@ export const restoreOwnedSkillsForUnbanBatchInternal = internalMutation({
|
||||
const nextSkill = { ...skill, ...patch };
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, nextSkill);
|
||||
|
||||
await setSkillEmbeddingsSoftDeleted(ctx, skill._id, false, now);
|
||||
restoredCount += 1;
|
||||
@@ -4519,7 +4842,7 @@ export const publishVersion: ReturnType<typeof action> = action({
|
||||
throw new ConvexError("MIT-0 license terms must be accepted to publish skills");
|
||||
}
|
||||
const { userId } = await requireUserFromAction(ctx);
|
||||
const target = (await ctx.runQuery(internal.publishers.resolvePublishTargetForUserInternal, {
|
||||
const target = (await ctx.runMutation(internal.publishers.resolvePublishTargetForUserInternal, {
|
||||
actorUserId: userId,
|
||||
ownerHandle: args.ownerHandle,
|
||||
minimumRole: "publisher",
|
||||
@@ -4727,6 +5050,7 @@ export const updateTags = mutation({
|
||||
changelogSource: version.changelogSource,
|
||||
clawdis: version.parsed?.clawdis,
|
||||
};
|
||||
patch.capabilityTags = version.capabilityTags;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -4750,6 +5074,38 @@ export const updateTags = mutation({
|
||||
},
|
||||
});
|
||||
|
||||
export const deleteTags = mutation({
|
||||
args: {
|
||||
skillId: v.id("skills"),
|
||||
tags: v.array(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const { user } = await requireUser(ctx);
|
||||
const skill = await ctx.db.get(args.skillId);
|
||||
if (!skill) throw new Error("Skill not found");
|
||||
if (skill.ownerUserId !== user._id) {
|
||||
assertModerator(user);
|
||||
}
|
||||
|
||||
const nextTags = { ...skill.tags };
|
||||
let changed = false;
|
||||
for (const tag of args.tags) {
|
||||
if (tag === "latest") continue; // protect the latest tag from deletion
|
||||
if (tag in nextTags) {
|
||||
delete nextTags[tag];
|
||||
changed = true;
|
||||
}
|
||||
}
|
||||
|
||||
if (!changed) return;
|
||||
|
||||
await ctx.db.patch(skill._id, {
|
||||
tags: nextTags,
|
||||
updatedAt: Date.now(),
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const setRedactionApproved = mutation({
|
||||
args: { skillId: v.id("skills"), approved: v.boolean() },
|
||||
handler: async (ctx, args) => {
|
||||
@@ -4924,7 +5280,11 @@ export const clearSkillManualOverride = mutation({
|
||||
});
|
||||
|
||||
export const setSoftDeleted = mutation({
|
||||
args: { skillId: v.id("skills"), deleted: v.boolean() },
|
||||
args: {
|
||||
skillId: v.id("skills"),
|
||||
deleted: v.boolean(),
|
||||
reason: v.optional(v.string()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const { user } = await requireUser(ctx);
|
||||
assertModerator(user);
|
||||
@@ -4932,9 +5292,16 @@ export const setSoftDeleted = mutation({
|
||||
if (!skill) throw new Error("Skill not found");
|
||||
|
||||
const now = Date.now();
|
||||
const note = args.reason ? trimManualOverrideNote(args.reason) : undefined;
|
||||
if (!note) {
|
||||
throw new ConvexError(
|
||||
args.deleted ? "Hide reason is required." : "Restore reason is required.",
|
||||
);
|
||||
}
|
||||
const patch: Partial<Doc<"skills">> = {
|
||||
softDeletedAt: args.deleted ? now : undefined,
|
||||
moderationStatus: args.deleted ? "hidden" : "active",
|
||||
moderationNotes: note,
|
||||
hiddenAt: args.deleted ? now : undefined,
|
||||
hiddenBy: args.deleted ? user._id : undefined,
|
||||
lastReviewedAt: now,
|
||||
@@ -4943,6 +5310,7 @@ export const setSoftDeleted = mutation({
|
||||
const nextSkill = { ...skill, ...patch };
|
||||
await ctx.db.patch(skill._id, patch);
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, skill, nextSkill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, nextSkill);
|
||||
|
||||
await setSkillEmbeddingsSoftDeleted(ctx, skill._id, args.deleted, now);
|
||||
|
||||
@@ -4951,7 +5319,11 @@ export const setSoftDeleted = mutation({
|
||||
action: args.deleted ? "skill.delete" : "skill.undelete",
|
||||
targetType: "skill",
|
||||
targetId: skill._id,
|
||||
metadata: { slug: skill.slug, softDeletedAt: args.deleted ? now : null },
|
||||
metadata: {
|
||||
slug: skill.slug,
|
||||
softDeletedAt: args.deleted ? now : null,
|
||||
reason: note,
|
||||
},
|
||||
createdAt: now,
|
||||
});
|
||||
},
|
||||
@@ -4977,6 +5349,7 @@ export const changeOwner = mutation({
|
||||
lastReviewedAt: now,
|
||||
updatedAt: now,
|
||||
});
|
||||
await adjustUserSkillStatsForSkillChange(ctx, skill, { ...skill, ownerUserId: args.ownerUserId });
|
||||
|
||||
const embeddings = await listSkillEmbeddingsForSkill(ctx, skill._id);
|
||||
for (const embedding of embeddings) {
|
||||
@@ -5630,6 +6003,72 @@ export const setDeprecatedBadge = mutation({
|
||||
},
|
||||
});
|
||||
|
||||
export const setSkillCapabilityTags = mutation({
|
||||
args: { skillId: v.id("skills"), capabilityTags: v.array(v.string()) },
|
||||
handler: async (ctx, args) => {
|
||||
const { user } = await requireUser(ctx);
|
||||
assertModerator(user);
|
||||
|
||||
const skill = await ctx.db.get(args.skillId);
|
||||
if (!skill) throw new Error("Skill not found");
|
||||
|
||||
const invalidTags = args.capabilityTags.filter(
|
||||
(tag) => !SKILL_CAPABILITY_TAGS.includes(tag as (typeof SKILL_CAPABILITY_TAGS)[number]),
|
||||
);
|
||||
if (invalidTags.length > 0) {
|
||||
throw new ConvexError(`Unknown capability tags: ${invalidTags.join(", ")}`);
|
||||
}
|
||||
|
||||
const selectedTags = new Set(args.capabilityTags);
|
||||
const normalizedTags = SKILL_CAPABILITY_TAGS.filter((tag) => selectedTags.has(tag));
|
||||
const now = Date.now();
|
||||
|
||||
if (skill.latestVersionId) {
|
||||
const latestVersion = await ctx.db.get(skill.latestVersionId);
|
||||
if (latestVersion) {
|
||||
await ctx.db.patch(latestVersion._id, {
|
||||
capabilityTags: normalizedTags.length ? normalizedTags : undefined,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
const nextSkill = {
|
||||
...skill,
|
||||
capabilityTags: normalizedTags.length ? normalizedTags : undefined,
|
||||
lastReviewedAt: now,
|
||||
updatedAt: now,
|
||||
};
|
||||
|
||||
await ctx.db.patch(skill._id, {
|
||||
capabilityTags: nextSkill.capabilityTags,
|
||||
lastReviewedAt: now,
|
||||
updatedAt: now,
|
||||
});
|
||||
|
||||
const owner = await getOwnerPublisher(ctx, {
|
||||
ownerPublisherId: nextSkill.ownerPublisherId,
|
||||
ownerUserId: nextSkill.ownerUserId,
|
||||
});
|
||||
await upsertSkillSearchDigest(ctx, {
|
||||
...extractDigestFields(nextSkill),
|
||||
ownerHandle: owner?.handle ?? "",
|
||||
ownerKind: owner?.kind,
|
||||
ownerName: owner?.linkedUserId ? owner.handle : undefined,
|
||||
ownerDisplayName: owner?.displayName,
|
||||
ownerImage: owner?.image,
|
||||
});
|
||||
|
||||
await ctx.db.insert("auditLogs", {
|
||||
actorUserId: user._id,
|
||||
action: "skill.capability_tags.set",
|
||||
targetType: "skill",
|
||||
targetId: skill._id,
|
||||
metadata: { capabilityTags: normalizedTags },
|
||||
createdAt: now,
|
||||
});
|
||||
},
|
||||
});
|
||||
|
||||
export const hardDelete = mutation({
|
||||
args: { skillId: v.id("skills") },
|
||||
handler: async (ctx, args) => {
|
||||
@@ -5691,6 +6130,7 @@ export const insertVersion = internalMutation({
|
||||
clawdis: v.optional(v.any()),
|
||||
license: v.optional(v.literal(PLATFORM_SKILL_LICENSE)),
|
||||
}),
|
||||
capabilityTags: v.optional(v.array(v.string())),
|
||||
summary: v.optional(v.string()),
|
||||
qualityAssessment: v.optional(
|
||||
v.object({
|
||||
@@ -5942,6 +6382,7 @@ export const insertVersion = internalMutation({
|
||||
forkOf,
|
||||
latestVersionId: undefined,
|
||||
tags: {},
|
||||
capabilityTags: args.capabilityTags,
|
||||
softDeletedAt: undefined,
|
||||
badges: {
|
||||
redactionApproved: undefined,
|
||||
@@ -5989,6 +6430,7 @@ export const insertVersion = internalMutation({
|
||||
// Digest sync is handled after the version patch below (line ~4222),
|
||||
// which captures the final state including latestVersionId and tags.
|
||||
await adjustGlobalPublicCountForSkillChange(ctx, null, skill);
|
||||
await adjustUserSkillStatsForSkillChange(ctx, null, skill);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -6010,6 +6452,7 @@ export const insertVersion = internalMutation({
|
||||
changelogSource: args.changelogSource,
|
||||
files: args.files,
|
||||
parsed: args.parsed,
|
||||
capabilityTags: args.capabilityTags,
|
||||
staticScan: args.staticScan,
|
||||
createdBy: userId,
|
||||
createdAt: now,
|
||||
@@ -6055,6 +6498,7 @@ export const insertVersion = internalMutation({
|
||||
clawdis: args.parsed.clawdis,
|
||||
},
|
||||
tags: nextTags,
|
||||
capabilityTags: args.capabilityTags,
|
||||
stats: { ...skill.stats, versions: skill.stats.versions + 1 },
|
||||
softDeletedAt: undefined,
|
||||
moderationStatus: initialModerationStatus,
|
||||
|
||||
+84
-3
@@ -1,5 +1,5 @@
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
import { getSoulBySlugInternal, insertVersion } from "./souls";
|
||||
import { getSoulBySlugInternal, insertVersion, list } from "./souls";
|
||||
|
||||
type WrappedHandler<TArgs> = {
|
||||
_handler: (ctx: unknown, args: TArgs) => Promise<unknown>;
|
||||
@@ -10,6 +10,7 @@ const insertVersionHandler = (insertVersion as unknown as WrappedHandler<Record<
|
||||
const getSoulBySlugInternalHandler = (
|
||||
getSoulBySlugInternal as unknown as WrappedHandler<{ slug: string }>
|
||||
)._handler;
|
||||
const listHandler = (list as unknown as WrappedHandler<{ ownerUserId?: string; limit?: number }>)._handler;
|
||||
|
||||
describe("souls.insertVersion", () => {
|
||||
it("throws a soul-specific ownership error for non-owners", async () => {
|
||||
@@ -23,7 +24,10 @@ describe("souls.insertVersion", () => {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table !== "souls") throw new Error(`unexpected table ${table}`);
|
||||
return {
|
||||
withIndex: (name: string, build: ((q: { eq: (field: string, value: string) => unknown }) => unknown) | undefined) => {
|
||||
withIndex: (
|
||||
name: string,
|
||||
build: ((q: { eq: (field: string, value: string) => unknown }) => unknown) | undefined,
|
||||
) => {
|
||||
if (name !== "by_slug") throw new Error(`unexpected index ${name}`);
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
@@ -92,7 +96,12 @@ describe("souls.insertVersion", () => {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table !== "souls") throw new Error(`unexpected table ${table}`);
|
||||
return {
|
||||
withIndex: (name: string, build: ((q: { eq: (field: string, value: string) => unknown }) => unknown) | undefined) => {
|
||||
withIndex: (
|
||||
name: string,
|
||||
build:
|
||||
| ((q: { eq: (field: string, value: string) => unknown }) => unknown)
|
||||
| undefined,
|
||||
) => {
|
||||
if (name !== "by_slug") throw new Error(`unexpected index ${name}`);
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
@@ -131,3 +140,75 @@ describe("souls.insertVersion", () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe("souls.list", () => {
|
||||
it("uses the active browse index and only takes the requested limit", async () => {
|
||||
let requestedIndex: string | null = null;
|
||||
let requestedSoftDeletedAt: number | undefined;
|
||||
let requestedLimit: number | null = null;
|
||||
|
||||
const result = await listHandler(
|
||||
{
|
||||
db: {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table !== "souls") throw new Error(`unexpected table ${table}`);
|
||||
return {
|
||||
withIndex: (
|
||||
name: string,
|
||||
build:
|
||||
| ((q: { eq: (field: string, value: undefined) => unknown }) => unknown)
|
||||
| undefined,
|
||||
) => {
|
||||
requestedIndex = name;
|
||||
const q = {
|
||||
eq: (field: string, value: undefined) => {
|
||||
if (field !== "softDeletedAt") throw new Error(`unexpected field ${field}`);
|
||||
requestedSoftDeletedAt = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
build?.(q);
|
||||
return {
|
||||
order: () => ({
|
||||
take: async (limit: number) => {
|
||||
requestedLimit = limit;
|
||||
return [
|
||||
{
|
||||
_id: "souls:1",
|
||||
_creationTime: 1,
|
||||
slug: "demo-soul",
|
||||
displayName: "Demo Soul",
|
||||
summary: "A demo soul",
|
||||
ownerUserId: "users:owner",
|
||||
ownerPublisherId: undefined,
|
||||
latestVersionId: undefined,
|
||||
tags: {},
|
||||
softDeletedAt: undefined,
|
||||
stats: { downloads: 1, stars: 2, versions: 3, comments: 4 },
|
||||
createdAt: 1,
|
||||
updatedAt: 2,
|
||||
},
|
||||
];
|
||||
},
|
||||
}),
|
||||
};
|
||||
},
|
||||
};
|
||||
}),
|
||||
},
|
||||
} as never,
|
||||
{ limit: 7 } as never,
|
||||
);
|
||||
|
||||
expect(requestedIndex).toBe("by_active_updated");
|
||||
expect(requestedSoftDeletedAt).toBeUndefined();
|
||||
expect(requestedLimit).toBe(7);
|
||||
expect(result).toEqual([
|
||||
expect.objectContaining({
|
||||
_id: "souls:1",
|
||||
slug: "demo-soul",
|
||||
displayName: "Demo Soul",
|
||||
}),
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
+2
-3
@@ -138,11 +138,10 @@ export const list = query({
|
||||
}
|
||||
const entries = await ctx.db
|
||||
.query("souls")
|
||||
.withIndex("by_active_updated", (q) => q.eq("softDeletedAt", undefined))
|
||||
.order("desc")
|
||||
.take(limit * 5);
|
||||
.take(limit);
|
||||
return entries
|
||||
.filter((soul) => !soul.softDeletedAt)
|
||||
.slice(0, limit)
|
||||
.map((soul) => toPublicSoul(soul))
|
||||
.filter((soul): soul is NonNullable<typeof soul> => Boolean(soul));
|
||||
},
|
||||
|
||||
+3
-1
@@ -34,6 +34,8 @@ export const toggle = mutation({
|
||||
return { starred: false };
|
||||
}
|
||||
|
||||
if (skill.softDeletedAt) throw new Error("Skill not found");
|
||||
|
||||
await ctx.db.insert("stars", {
|
||||
skillId: args.skillId,
|
||||
userId,
|
||||
@@ -70,7 +72,7 @@ export const addStarInternal = internalMutation({
|
||||
args: { userId: v.id("users"), skillId: v.id("skills") },
|
||||
handler: async (ctx, args) => {
|
||||
const skill = await ctx.db.get(args.skillId);
|
||||
if (!skill) throw new Error("Skill not found");
|
||||
if (!skill || skill.softDeletedAt) throw new Error("Skill not found");
|
||||
const existing = await ctx.db
|
||||
.query("stars")
|
||||
.withIndex("by_skill_user", (q) => q.eq("skillId", args.skillId).eq("userId", args.userId))
|
||||
|
||||
@@ -0,0 +1,303 @@
|
||||
/* @vitest-environment node */
|
||||
import { describe, expect, it, vi } from "vitest";
|
||||
|
||||
// Mock the Convex function wrappers so that importing statsMaintenance.ts does
|
||||
// not attempt to load the Convex runtime (convex/server) in the Node test env.
|
||||
vi.mock("./functions", () => ({
|
||||
internalMutation: (def: { handler: unknown }) => def,
|
||||
internalQuery: (def: { handler: unknown }) => def,
|
||||
internalAction: (def: { handler: unknown }) => def,
|
||||
}));
|
||||
|
||||
vi.mock("./_generated/api", () => ({
|
||||
internal: {
|
||||
statsMaintenance: {
|
||||
backfillSkillStatFieldsInternal: Symbol("backfillSkillStatFieldsInternal"),
|
||||
getSkillStatBackfillStateInternal: Symbol("getSkillStatBackfillStateInternal"),
|
||||
setSkillStatBackfillStateInternal: Symbol("setSkillStatBackfillStateInternal"),
|
||||
reconcileSkillStarCounts: Symbol("reconcileSkillStarCounts"),
|
||||
},
|
||||
},
|
||||
}));
|
||||
|
||||
const { __test, reconcileSkillStarCountsHandler } = await import("./statsMaintenance");
|
||||
const { buildSkillStatPatch } = __test;
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Helpers
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
/**
|
||||
* Build a minimal skill doc for testing. Only the stat-related fields are
|
||||
* required; everything else is left as `undefined` / cast via `as never`.
|
||||
*/
|
||||
function makeSkill(overrides: {
|
||||
statsDownloads?: number;
|
||||
statsStars?: number;
|
||||
statsInstallsCurrent?: number;
|
||||
statsInstallsAllTime?: number;
|
||||
stats: {
|
||||
downloads: number;
|
||||
stars: number;
|
||||
installsCurrent?: number;
|
||||
installsAllTime?: number;
|
||||
comments: number;
|
||||
};
|
||||
}) {
|
||||
return overrides as never;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// buildSkillStatPatch
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe("buildSkillStatPatch", () => {
|
||||
it("scenario 1: top-level fields present and already in sync with nested → returns null", () => {
|
||||
const skill = makeSkill({
|
||||
statsDownloads: 10,
|
||||
statsStars: 5,
|
||||
statsInstallsCurrent: 3,
|
||||
statsInstallsAllTime: 20,
|
||||
stats: { downloads: 10, stars: 5, installsCurrent: 3, installsAllTime: 20, comments: 1 },
|
||||
});
|
||||
|
||||
expect(buildSkillStatPatch(skill)).toBeNull();
|
||||
});
|
||||
|
||||
it("scenario 2: top-level fields present but nested fields are stale → patches nested to match top-level", () => {
|
||||
const skill = makeSkill({
|
||||
statsDownloads: 10,
|
||||
statsStars: 5,
|
||||
statsInstallsCurrent: 3,
|
||||
statsInstallsAllTime: 20,
|
||||
stats: { downloads: 1, stars: 1, installsCurrent: 0, installsAllTime: 0, comments: 0 },
|
||||
});
|
||||
|
||||
const patch = buildSkillStatPatch(skill);
|
||||
expect(patch).not.toBeNull();
|
||||
// Top-level fields must be written with the canonical (top-level) values.
|
||||
expect(patch!.statsDownloads).toBe(10);
|
||||
expect(patch!.statsStars).toBe(5);
|
||||
expect(patch!.statsInstallsCurrent).toBe(3);
|
||||
expect(patch!.statsInstallsAllTime).toBe(20);
|
||||
// Nested fields must be brought in sync with the top-level values.
|
||||
expect(patch!.stats.downloads).toBe(10);
|
||||
expect(patch!.stats.stars).toBe(5);
|
||||
expect(patch!.stats.installsCurrent).toBe(3);
|
||||
expect(patch!.stats.installsAllTime).toBe(20);
|
||||
});
|
||||
|
||||
it("scenario 3: top-level fields absent (pre-migration doc) → reads from nested, writes both sets", () => {
|
||||
const skill = makeSkill({
|
||||
// No statsDownloads / statsStars / etc. — pre-migration document.
|
||||
stats: { downloads: 7, stars: 3, installsCurrent: 2, installsAllTime: 15, comments: 4 },
|
||||
});
|
||||
|
||||
const patch = buildSkillStatPatch(skill);
|
||||
expect(patch).not.toBeNull();
|
||||
// Top-level fields must be populated from the nested values.
|
||||
expect(patch!.statsDownloads).toBe(7);
|
||||
expect(patch!.statsStars).toBe(3);
|
||||
expect(patch!.statsInstallsCurrent).toBe(2);
|
||||
expect(patch!.statsInstallsAllTime).toBe(15);
|
||||
// Nested fields must remain consistent.
|
||||
expect(patch!.stats.downloads).toBe(7);
|
||||
expect(patch!.stats.stars).toBe(3);
|
||||
expect(patch!.stats.installsCurrent).toBe(2);
|
||||
expect(patch!.stats.installsAllTime).toBe(15);
|
||||
});
|
||||
|
||||
it("scenario 4: top-level fields present but nested is out of sync → patches nested to match top-level (not the other way around)", () => {
|
||||
// This is the exact bug that was previously shipped: the old code wrote
|
||||
// nested → top-level instead of top-level → nested.
|
||||
const skill = makeSkill({
|
||||
statsDownloads: 100,
|
||||
statsStars: 50,
|
||||
statsInstallsCurrent: 30,
|
||||
statsInstallsAllTime: 200,
|
||||
stats: { downloads: 1, stars: 1, installsCurrent: 1, installsAllTime: 1, comments: 0 },
|
||||
});
|
||||
|
||||
const patch = buildSkillStatPatch(skill);
|
||||
expect(patch).not.toBeNull();
|
||||
// The canonical top-level values must win.
|
||||
expect(patch!.statsDownloads).toBe(100);
|
||||
expect(patch!.statsStars).toBe(50);
|
||||
expect(patch!.statsInstallsCurrent).toBe(30);
|
||||
expect(patch!.statsInstallsAllTime).toBe(200);
|
||||
// The stale nested values must be overwritten by the top-level values.
|
||||
expect(patch!.stats.downloads).toBe(100);
|
||||
expect(patch!.stats.stars).toBe(50);
|
||||
expect(patch!.stats.installsCurrent).toBe(30);
|
||||
expect(patch!.stats.installsAllTime).toBe(200);
|
||||
});
|
||||
|
||||
it("preserves unrelated nested fields (e.g. comments) when patching stat fields", () => {
|
||||
const skill = makeSkill({
|
||||
statsDownloads: 5,
|
||||
statsStars: 2,
|
||||
statsInstallsCurrent: 1,
|
||||
statsInstallsAllTime: 10,
|
||||
stats: { downloads: 0, stars: 0, installsCurrent: 0, installsAllTime: 0, comments: 99 },
|
||||
});
|
||||
|
||||
const patch = buildSkillStatPatch(skill);
|
||||
expect(patch).not.toBeNull();
|
||||
// comments is not a stat field managed by buildSkillStatPatch — it must be
|
||||
// carried over unchanged from the original nested object.
|
||||
expect(patch!.stats.comments).toBe(99);
|
||||
});
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// reconcileSkillStarCountsHandler
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
describe("reconcileSkillStarCounts", () => {
|
||||
/**
|
||||
* Build a minimal db mock that returns a single-page result for skills and
|
||||
* configurable star / comment record counts.
|
||||
*/
|
||||
function makeCtx(options: {
|
||||
skill: {
|
||||
_id: string;
|
||||
statsStars?: number;
|
||||
stats: { stars: number; comments: number };
|
||||
softDeletedAt?: number;
|
||||
};
|
||||
actualStarCount: number;
|
||||
actualCommentCount: number;
|
||||
}) {
|
||||
const { skill, actualStarCount, actualCommentCount } = options;
|
||||
|
||||
const starRecords = Array.from({ length: actualStarCount }, (_, i) => ({
|
||||
_id: `stars:${i}`,
|
||||
skillId: skill._id,
|
||||
}));
|
||||
|
||||
const commentRecords = Array.from({ length: actualCommentCount }, (_, i) => ({
|
||||
_id: `comments:${i}`,
|
||||
skillId: skill._id,
|
||||
softDeletedAt: undefined,
|
||||
}));
|
||||
|
||||
const paginate = vi.fn().mockResolvedValue({
|
||||
page: [skill],
|
||||
continueCursor: null,
|
||||
isDone: true,
|
||||
});
|
||||
|
||||
const collect = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(starRecords)
|
||||
.mockResolvedValueOnce(commentRecords);
|
||||
|
||||
const withIndex = vi.fn().mockReturnValue({ collect });
|
||||
|
||||
const patch = vi.fn().mockResolvedValue(undefined);
|
||||
|
||||
const ctx = {
|
||||
db: {
|
||||
query: vi.fn().mockReturnValue({
|
||||
order: vi.fn().mockReturnValue({ paginate }),
|
||||
withIndex,
|
||||
}),
|
||||
patch,
|
||||
},
|
||||
} as never;
|
||||
|
||||
return { ctx, patch };
|
||||
}
|
||||
|
||||
it("reads from top-level statsStars (canonical path) when deciding whether to patch", async () => {
|
||||
// statsStars is correct (matches actual count), but stats.stars is stale.
|
||||
// The reconcile job uses the canonical read path (top-level preferred), so
|
||||
// it should NOT trigger a patch based on the star count alone.
|
||||
const skill = {
|
||||
_id: "skills:1",
|
||||
statsStars: 5, // canonical value — correct
|
||||
stats: { stars: 99, comments: 0 }, // legacy value — stale, but not reconcile's concern
|
||||
};
|
||||
|
||||
const { ctx, patch } = makeCtx({ skill, actualStarCount: 5, actualCommentCount: 0 });
|
||||
|
||||
const result = await reconcileSkillStarCountsHandler(ctx, {});
|
||||
|
||||
expect(result.scanned).toBe(1);
|
||||
expect(result.patched).toBe(0);
|
||||
expect(patch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("falls back to stats.stars when statsStars is absent (pre-migration doc)", async () => {
|
||||
// Pre-migration doc: no top-level statsStars. The canonical read path
|
||||
// falls back to stats.stars. If that also matches actual count, no patch.
|
||||
const skill = {
|
||||
_id: "skills:1",
|
||||
// statsStars intentionally absent
|
||||
stats: { stars: 3, comments: 0 },
|
||||
};
|
||||
|
||||
const { ctx, patch } = makeCtx({ skill, actualStarCount: 3, actualCommentCount: 0 });
|
||||
|
||||
const result = await reconcileSkillStarCountsHandler(ctx, {});
|
||||
|
||||
expect(result.scanned).toBe(1);
|
||||
expect(result.patched).toBe(0);
|
||||
expect(patch).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("patches both statsStars and stats.stars when canonical value drifts from actual count", async () => {
|
||||
const skill = {
|
||||
_id: "skills:1",
|
||||
statsStars: 10, // canonical value — out of sync with actual
|
||||
stats: { stars: 10, comments: 0 },
|
||||
};
|
||||
|
||||
const { ctx, patch } = makeCtx({ skill, actualStarCount: 7, actualCommentCount: 0 });
|
||||
|
||||
const result = await reconcileSkillStarCountsHandler(ctx, {});
|
||||
|
||||
expect(result.scanned).toBe(1);
|
||||
expect(result.patched).toBe(1);
|
||||
expect(patch).toHaveBeenCalledWith("skills:1", expect.objectContaining({
|
||||
statsStars: 7,
|
||||
stats: expect.objectContaining({ stars: 7 }),
|
||||
}));
|
||||
});
|
||||
|
||||
it("patches when comment count drifts even if star count is correct", async () => {
|
||||
const skill = {
|
||||
_id: "skills:1",
|
||||
statsStars: 5,
|
||||
stats: { stars: 5, comments: 10 }, // comments out of sync
|
||||
};
|
||||
|
||||
const { ctx, patch } = makeCtx({ skill, actualStarCount: 5, actualCommentCount: 3 });
|
||||
|
||||
const result = await reconcileSkillStarCountsHandler(ctx, {});
|
||||
|
||||
expect(result.scanned).toBe(1);
|
||||
expect(result.patched).toBe(1);
|
||||
expect(patch).toHaveBeenCalledWith("skills:1", expect.objectContaining({
|
||||
stats: expect.objectContaining({ comments: 3 }),
|
||||
}));
|
||||
});
|
||||
|
||||
it("skips soft-deleted skills", async () => {
|
||||
const skill = {
|
||||
_id: "skills:1",
|
||||
softDeletedAt: 12345,
|
||||
statsStars: 0,
|
||||
stats: { stars: 0, comments: 0 },
|
||||
};
|
||||
|
||||
const { ctx, patch } = makeCtx({ skill, actualStarCount: 5, actualCommentCount: 0 });
|
||||
|
||||
const result = await reconcileSkillStarCountsHandler(ctx, {});
|
||||
|
||||
// Soft-deleted skills are excluded from scanned count and never patched.
|
||||
expect(result.scanned).toBe(0);
|
||||
expect(result.patched).toBe(0);
|
||||
expect(patch).not.toHaveBeenCalled();
|
||||
});
|
||||
});
|
||||
+107
-55
@@ -183,25 +183,53 @@ export const runSkillStatBackfillInternal: ReturnType<typeof internalAction> = i
|
||||
|
||||
function buildSkillStatPatch(skill: Doc<"skills">) {
|
||||
const stats = skill.stats;
|
||||
const nextDownloads = stats.downloads;
|
||||
const nextStars = stats.stars;
|
||||
const nextInstallsCurrent = stats.installsCurrent ?? 0;
|
||||
const nextInstallsAllTime = stats.installsAllTime ?? 0;
|
||||
|
||||
if (
|
||||
// Prefer the top-level stat fields when they exist (they are kept up-to-date
|
||||
// by applySkillStatDeltas on every event flush). Fall back to the legacy
|
||||
// nested `stats` object only for documents that pre-date the migration.
|
||||
const nextDownloads =
|
||||
typeof skill.statsDownloads === "number" ? skill.statsDownloads : stats.downloads;
|
||||
const nextStars =
|
||||
typeof skill.statsStars === "number" ? skill.statsStars : stats.stars;
|
||||
const nextInstallsCurrent =
|
||||
typeof skill.statsInstallsCurrent === "number"
|
||||
? skill.statsInstallsCurrent
|
||||
: (stats.installsCurrent ?? 0);
|
||||
const nextInstallsAllTime =
|
||||
typeof skill.statsInstallsAllTime === "number"
|
||||
? skill.statsInstallsAllTime
|
||||
: (stats.installsAllTime ?? 0);
|
||||
|
||||
// Check whether both sets of fields are already in sync.
|
||||
const topLevelInSync =
|
||||
skill.statsDownloads === nextDownloads &&
|
||||
skill.statsStars === nextStars &&
|
||||
skill.statsInstallsCurrent === nextInstallsCurrent &&
|
||||
skill.statsInstallsAllTime === nextInstallsAllTime
|
||||
) {
|
||||
skill.statsInstallsAllTime === nextInstallsAllTime;
|
||||
|
||||
const nestedInSync =
|
||||
stats.downloads === nextDownloads &&
|
||||
stats.stars === nextStars &&
|
||||
(stats.installsCurrent ?? 0) === nextInstallsCurrent &&
|
||||
(stats.installsAllTime ?? 0) === nextInstallsAllTime;
|
||||
|
||||
if (topLevelInSync && nestedInSync) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Write both sets of fields so they stay in sync.
|
||||
return {
|
||||
statsDownloads: nextDownloads,
|
||||
statsStars: nextStars,
|
||||
statsInstallsCurrent: nextInstallsCurrent,
|
||||
statsInstallsAllTime: nextInstallsAllTime,
|
||||
stats: {
|
||||
...stats,
|
||||
downloads: nextDownloads,
|
||||
stars: nextStars,
|
||||
installsCurrent: nextInstallsCurrent,
|
||||
installsAllTime: nextInstallsAllTime,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
@@ -215,60 +243,79 @@ function buildSkillStatPatch(skill: Doc<"skills">) {
|
||||
*
|
||||
* Downloads and installs are event-sourced only (no separate table to count from),
|
||||
* so they cannot be reconciled this way.
|
||||
*
|
||||
* Exported as a standalone function so it can be unit-tested directly without
|
||||
* going through the Convex internalMutation wrapper.
|
||||
*/
|
||||
export async function reconcileSkillStarCountsHandler(
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
ctx: { db: { query: any; patch: any } },
|
||||
args: { cursor?: string; batchSize?: number },
|
||||
) {
|
||||
const batchSize = clampInt(args.batchSize ?? 50, 1, 200);
|
||||
const now = Date.now();
|
||||
|
||||
const { page, isDone, continueCursor } = await ctx.db
|
||||
.query("skills")
|
||||
.order("asc")
|
||||
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
|
||||
|
||||
let scanned = 0;
|
||||
let patched = 0;
|
||||
for (const skill of page) {
|
||||
if (skill.softDeletedAt) continue;
|
||||
scanned += 1;
|
||||
// Count actual star records for this skill
|
||||
const starRecords = await ctx.db
|
||||
.query("stars")
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
.withIndex("by_skill_user", (q: any) => q.eq("skillId", skill._id))
|
||||
.collect();
|
||||
const actualStars = starRecords.length;
|
||||
|
||||
// Count actual comment records for this skill
|
||||
const commentRecords = await ctx.db
|
||||
.query("comments")
|
||||
// eslint-disable-next-line @typescript-eslint/no-explicit-any
|
||||
.withIndex("by_skill", (q: any) => q.eq("skillId", skill._id))
|
||||
.collect();
|
||||
const actualComments = commentRecords.filter((c: { softDeletedAt?: unknown }) => !c.softDeletedAt).length;
|
||||
|
||||
// Check if stats are out of sync (compare against the canonical value
|
||||
// used by toPublicSkill: prefer top-level field, fall back to nested).
|
||||
const currentStars =
|
||||
typeof skill.statsStars === "number" ? skill.statsStars : skill.stats.stars;
|
||||
|
||||
if (currentStars !== actualStars || skill.stats.comments !== actualComments) {
|
||||
const updatedStats = {
|
||||
...skill.stats,
|
||||
stars: actualStars,
|
||||
comments: actualComments,
|
||||
};
|
||||
// Keep both the top-level index field and the legacy nested field in sync.
|
||||
await ctx.db.patch(skill._id, {
|
||||
statsStars: actualStars,
|
||||
stats: updatedStats,
|
||||
updatedAt: now,
|
||||
});
|
||||
patched += 1;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
scanned,
|
||||
patched,
|
||||
cursor: isDone ? null : continueCursor,
|
||||
isDone,
|
||||
};
|
||||
}
|
||||
|
||||
export const reconcileSkillStarCounts = internalMutation({
|
||||
args: {
|
||||
cursor: v.optional(v.string()),
|
||||
batchSize: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const batchSize = clampInt(args.batchSize ?? 50, 1, 200);
|
||||
const now = Date.now();
|
||||
|
||||
const { page, isDone, continueCursor } = await ctx.db
|
||||
.query("skills")
|
||||
.order("asc")
|
||||
.paginate({ cursor: args.cursor ?? null, numItems: batchSize });
|
||||
|
||||
let patched = 0;
|
||||
for (const skill of page) {
|
||||
// Count actual star records for this skill
|
||||
const starRecords = await ctx.db
|
||||
.query("stars")
|
||||
.withIndex("by_skill_user", (q) => q.eq("skillId", skill._id))
|
||||
.collect();
|
||||
const actualStars = starRecords.length;
|
||||
|
||||
// Count actual comment records for this skill
|
||||
const commentRecords = await ctx.db
|
||||
.query("comments")
|
||||
.withIndex("by_skill", (q) => q.eq("skillId", skill._id))
|
||||
.collect();
|
||||
const actualComments = commentRecords.filter((c) => !c.softDeletedAt).length;
|
||||
|
||||
// Check if stats are out of sync
|
||||
if (skill.stats.stars !== actualStars || skill.stats.comments !== actualComments) {
|
||||
const updatedStats = {
|
||||
...skill.stats,
|
||||
stars: actualStars,
|
||||
comments: actualComments,
|
||||
};
|
||||
await ctx.db.patch(skill._id, {
|
||||
statsStars: actualStars,
|
||||
stats: updatedStats,
|
||||
updatedAt: now,
|
||||
});
|
||||
patched += 1;
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
scanned: page.length,
|
||||
patched,
|
||||
cursor: isDone ? null : continueCursor,
|
||||
isDone,
|
||||
};
|
||||
},
|
||||
handler: reconcileSkillStarCountsHandler,
|
||||
});
|
||||
|
||||
export const runReconcileSkillStarCountsInternal = internalAction({
|
||||
@@ -305,6 +352,11 @@ function clampInt(value: number, min: number, max: number) {
|
||||
return Math.min(Math.max(value, min), max);
|
||||
}
|
||||
|
||||
// Exported for unit testing only — not part of the public API.
|
||||
export const __test = {
|
||||
buildSkillStatPatch,
|
||||
};
|
||||
|
||||
/**
|
||||
* Count a page of skillSearchDigest docs and return the partial public count.
|
||||
* Each query runs in its own transaction (~1000 docs, ~900 KB), well under limits.
|
||||
|
||||
+585
-6
@@ -18,6 +18,7 @@ const { getAuthUserId } = await import("@convex-dev/auth/server");
|
||||
const { insertStatEvent } = await import("./skillStatEvents");
|
||||
const {
|
||||
ensureHandler,
|
||||
getByHandle,
|
||||
list,
|
||||
searchInternal,
|
||||
banUserInternal,
|
||||
@@ -32,6 +33,8 @@ type WrappedHandler<TArgs, TResult> = {
|
||||
};
|
||||
|
||||
const meHandler = (me as unknown as WrappedHandler<Record<string, never>, unknown>)._handler;
|
||||
const getByHandleHandler = (getByHandle as unknown as WrappedHandler<{ handle: string }, unknown>)
|
||||
._handler;
|
||||
|
||||
function makeCtx() {
|
||||
const patch = vi.fn();
|
||||
@@ -116,12 +119,63 @@ function makeCtx() {
|
||||
};
|
||||
}
|
||||
|
||||
function makeListCtx(users: Array<Record<string, unknown>>) {
|
||||
function makeListCtx(
|
||||
users: Array<Record<string, unknown>>,
|
||||
options?: {
|
||||
publishersByHandle?: Record<string, Record<string, unknown>>;
|
||||
usersById?: Record<string, Record<string, unknown> | null>;
|
||||
},
|
||||
) {
|
||||
const take = vi.fn(async (n: number) => users.slice(0, n));
|
||||
const collect = vi.fn(async () => users);
|
||||
const order = vi.fn(() => ({ take, collect }));
|
||||
const query = vi.fn(() => ({ order }));
|
||||
const get = vi.fn();
|
||||
const publishersByHandle = options?.publishersByHandle ?? {};
|
||||
const usersById = options?.usersById ?? {};
|
||||
const query = vi.fn((table: string) => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
order,
|
||||
withIndex: (
|
||||
name: string,
|
||||
cb?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
if (name !== "handle") throw new Error(`Unexpected users index ${name}`);
|
||||
let handle = "";
|
||||
cb?.({
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return {};
|
||||
},
|
||||
});
|
||||
return {
|
||||
unique: vi.fn(async () => users.find((user) => user.handle === handle) ?? null),
|
||||
};
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (
|
||||
name: string,
|
||||
cb?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
if (name !== "by_handle") throw new Error(`Unexpected publishers index ${name}`);
|
||||
let handle = "";
|
||||
cb?.({
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return {};
|
||||
},
|
||||
});
|
||||
return { unique: vi.fn(async () => publishersByHandle[handle] ?? null) };
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected table ${table}`);
|
||||
});
|
||||
const get = vi.fn<(id: string) => Promise<Record<string, unknown> | null>>(
|
||||
async (id: string) => usersById[id] ?? null,
|
||||
);
|
||||
return {
|
||||
ctx: { db: { query, get, normalizeId: vi.fn() } } as never,
|
||||
take,
|
||||
@@ -335,6 +389,115 @@ describe("ensureHandler", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("repairs an existing handle that is no longer claimable", async () => {
|
||||
const { ctx, patch, query } = makeCtx();
|
||||
query.mockImplementation(((table: string) => {
|
||||
if (table === "reservedHandles") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "by_handle_active_updatedAt") {
|
||||
throw new Error(`Unexpected reservedHandles index ${name}`);
|
||||
}
|
||||
return { order: () => ({ take: async () => [] }) };
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (
|
||||
name: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
if (field === "linkedUserId") linkedUserId = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
if (name === "by_handle") {
|
||||
return {
|
||||
unique: vi.fn(async () => {
|
||||
if (handle === "openclaw") {
|
||||
return {
|
||||
_id: "publishers:openclaw",
|
||||
kind: "org",
|
||||
handle: "openclaw",
|
||||
displayName: "OpenClaw",
|
||||
};
|
||||
}
|
||||
return null;
|
||||
}),
|
||||
};
|
||||
}
|
||||
if (name === "by_linked_user") {
|
||||
return {
|
||||
unique: vi.fn(async () =>
|
||||
linkedUserId === "users:owner"
|
||||
? {
|
||||
_id: "publishers:openclaw-user",
|
||||
kind: "user",
|
||||
handle: "openclaw-user",
|
||||
linkedUserId: "users:owner",
|
||||
displayName: "OpenClaw User",
|
||||
}
|
||||
: null,
|
||||
),
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected publishers index ${name}`);
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publisherMembers") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "by_publisher_user") {
|
||||
throw new Error(`Unexpected publisherMembers index ${name}`);
|
||||
}
|
||||
return { unique: vi.fn(async () => null) };
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "packages" || table === "skills") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "by_owner_publisher") {
|
||||
throw new Error(`Unexpected ${table} index ${name}`);
|
||||
}
|
||||
return { collect: vi.fn(async () => []) };
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected table ${table}`);
|
||||
}) as never);
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:owner",
|
||||
user: {
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "openclaw",
|
||||
displayName: "openclaw",
|
||||
name: "openclaw",
|
||||
email: "owner@example.com",
|
||||
role: "user",
|
||||
createdAt: 1,
|
||||
personalPublisherId: "publishers:openclaw-user",
|
||||
},
|
||||
} as never);
|
||||
|
||||
await ensureHandler(ctx);
|
||||
|
||||
expect(patch).toHaveBeenCalledWith("users:owner", {
|
||||
handle: "openclaw-2",
|
||||
displayName: "openclaw-2",
|
||||
updatedAt: expect.any(Number),
|
||||
});
|
||||
});
|
||||
|
||||
it("does not auto-claim a reserved handle for another user", async () => {
|
||||
const { ctx, patch, query } = makeCtx();
|
||||
query.mockImplementation(((table: string) => {
|
||||
@@ -395,7 +558,10 @@ describe("ensureHandler", () => {
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (name: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
withIndex: (
|
||||
name: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
@@ -498,6 +664,181 @@ describe("me", () => {
|
||||
expect(result).toBeNull();
|
||||
expect(get).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("returns null when auth resolves to an invalid user id", async () => {
|
||||
vi.mocked(getAuthUserId).mockResolvedValue("users:broken" as never);
|
||||
const get = vi.fn(async (id: string) => {
|
||||
if (id === "users:broken") throw new Error("Table mismatch");
|
||||
return null;
|
||||
});
|
||||
|
||||
const result = await meHandler({ db: { get } } as never, {});
|
||||
|
||||
expect(result).toBeNull();
|
||||
expect(get).toHaveBeenCalledWith("users:broken");
|
||||
});
|
||||
});
|
||||
|
||||
describe("users.getByHandle", () => {
|
||||
it("normalizes the incoming handle before querying", async () => {
|
||||
const unique = vi.fn(async () => ({
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "jaredforreal",
|
||||
name: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
image: undefined,
|
||||
bio: undefined,
|
||||
}));
|
||||
|
||||
const result = await getByHandleHandler(
|
||||
{
|
||||
db: {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table !== "users") throw new Error(`Unexpected table ${table}`);
|
||||
return {
|
||||
withIndex: (
|
||||
name: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
if (name !== "handle") throw new Error(`Unexpected index ${name}`);
|
||||
let handle = "";
|
||||
const q = {
|
||||
eq: (field: string, value: string) => {
|
||||
if (field === "handle") handle = value;
|
||||
return q;
|
||||
},
|
||||
};
|
||||
builder?.(q);
|
||||
expect(handle).toBe("jaredforreal");
|
||||
return { unique };
|
||||
},
|
||||
};
|
||||
}),
|
||||
get: vi.fn(),
|
||||
},
|
||||
} as never,
|
||||
{ handle: " @JaredForReal " },
|
||||
);
|
||||
|
||||
expect(unique).toHaveBeenCalledOnce();
|
||||
expect(result).toMatchObject({
|
||||
_id: "users:owner",
|
||||
handle: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
});
|
||||
});
|
||||
|
||||
it("falls back to the linked user for a personal publisher handle", async () => {
|
||||
const userUnique = vi.fn(async () => null);
|
||||
const publisherUnique = vi.fn(async () => ({
|
||||
_id: "publishers:jaredforreal",
|
||||
kind: "user",
|
||||
handle: "jaredforreal",
|
||||
linkedUserId: "users:owner",
|
||||
displayName: "Jared",
|
||||
}));
|
||||
const get = vi.fn(async (id: string) =>
|
||||
id === "users:owner"
|
||||
? {
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "jared",
|
||||
name: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
image: undefined,
|
||||
bio: "Profile",
|
||||
}
|
||||
: null,
|
||||
);
|
||||
|
||||
const result = await getByHandleHandler(
|
||||
{
|
||||
db: {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "handle") throw new Error(`Unexpected users index ${name}`);
|
||||
return { unique: userUnique };
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "by_handle") throw new Error(`Unexpected publishers index ${name}`);
|
||||
return { unique: publisherUnique };
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected table ${table}`);
|
||||
}),
|
||||
get,
|
||||
},
|
||||
} as never,
|
||||
{ handle: "jaredforreal" },
|
||||
);
|
||||
|
||||
expect(userUnique).toHaveBeenCalledOnce();
|
||||
expect(publisherUnique).toHaveBeenCalledOnce();
|
||||
expect(get).toHaveBeenCalledWith("users:owner");
|
||||
expect(result).toMatchObject({
|
||||
_id: "users:owner",
|
||||
handle: "jared",
|
||||
name: "jaredforreal",
|
||||
displayName: "Jared",
|
||||
bio: "Profile",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not resolve a deleted personal publisher handle", async () => {
|
||||
const userUnique = vi.fn(async () => null);
|
||||
const publisherUnique = vi.fn(async () => ({
|
||||
_id: "publishers:jaredforreal",
|
||||
kind: "user",
|
||||
handle: "jaredforreal",
|
||||
linkedUserId: "users:owner",
|
||||
deletedAt: 1_700_000_000_000,
|
||||
displayName: "Jared",
|
||||
}));
|
||||
const get = vi.fn(async () => {
|
||||
throw new Error("linked user should not be loaded for inactive publishers");
|
||||
});
|
||||
|
||||
const result = await getByHandleHandler(
|
||||
{
|
||||
db: {
|
||||
query: vi.fn((table: string) => {
|
||||
if (table === "users") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "handle") throw new Error(`Unexpected users index ${name}`);
|
||||
return { unique: userUnique };
|
||||
},
|
||||
};
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (name: string) => {
|
||||
if (name !== "by_handle") throw new Error(`Unexpected publishers index ${name}`);
|
||||
return { unique: publisherUnique };
|
||||
},
|
||||
};
|
||||
}
|
||||
throw new Error(`Unexpected table ${table}`);
|
||||
}),
|
||||
get,
|
||||
},
|
||||
} as never,
|
||||
{ handle: "jaredforreal" },
|
||||
);
|
||||
|
||||
expect(userUnique).toHaveBeenCalledOnce();
|
||||
expect(publisherUnique).toHaveBeenCalledOnce();
|
||||
expect(get).not.toHaveBeenCalled();
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("users.syncGitHubProfileInternal", () => {
|
||||
@@ -580,7 +921,10 @@ describe("users.syncGitHubProfileInternal", () => {
|
||||
}
|
||||
if (table === "publishers") {
|
||||
return {
|
||||
withIndex: (name: string, builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown) => {
|
||||
withIndex: (
|
||||
name: string,
|
||||
builder?: (q: { eq: (field: string, value: string) => unknown }) => unknown,
|
||||
) => {
|
||||
let handle = "";
|
||||
let linkedUserId = "";
|
||||
const q = {
|
||||
@@ -731,6 +1075,81 @@ describe("users.list", () => {
|
||||
expect(result.items[0]?.handle).toBe("alice");
|
||||
});
|
||||
|
||||
it("includes an exact older handle match outside the bounded scan", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:admin",
|
||||
user: { _id: "users:admin", role: "admin" },
|
||||
} as never);
|
||||
const users = [
|
||||
...Array.from({ length: 500 }, (_value, index) => ({
|
||||
_id: `users:recent-${index}`,
|
||||
_creationTime: 10_000 - index,
|
||||
handle: `recent-${index}`,
|
||||
role: "user",
|
||||
})),
|
||||
{ _id: "users:older", _creationTime: 1, handle: "alice", role: "user" },
|
||||
];
|
||||
const { ctx, take, collect } = makeListCtx(users);
|
||||
const listHandler = (
|
||||
list as unknown as { _handler: (ctx: unknown, args: unknown) => Promise<unknown> }
|
||||
)._handler;
|
||||
|
||||
const result = (await listHandler(ctx, { limit: 50, search: "alice" })) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
total: number;
|
||||
};
|
||||
|
||||
expect(take).toHaveBeenCalledWith(500);
|
||||
expect(collect).not.toHaveBeenCalled();
|
||||
expect(result.total).toBe(1);
|
||||
expect(result.items[0]?._id).toBe("users:older");
|
||||
});
|
||||
|
||||
it("includes an exact personal publisher handle match without a full collect", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:admin",
|
||||
user: { _id: "users:admin", role: "admin" },
|
||||
} as never);
|
||||
const users = [{ _id: "users:1", _creationTime: 2, handle: "alice", role: "user" }];
|
||||
const { ctx, take, collect } = makeListCtx(users, {
|
||||
publishersByHandle: {
|
||||
lmlukef: {
|
||||
_id: "publishers:lmlukef",
|
||||
kind: "user",
|
||||
handle: "lmlukef",
|
||||
linkedUserId: "users:owner",
|
||||
},
|
||||
},
|
||||
usersById: {
|
||||
"users:owner": {
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "luke",
|
||||
name: "different-gh-login",
|
||||
displayName: "Luke",
|
||||
role: "user",
|
||||
},
|
||||
},
|
||||
});
|
||||
const listHandler = (
|
||||
list as unknown as { _handler: (ctx: unknown, args: unknown) => Promise<unknown> }
|
||||
)._handler;
|
||||
|
||||
const result = (await listHandler(ctx, { limit: 50, search: "lmLukeF" })) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
total: number;
|
||||
};
|
||||
|
||||
expect(take).toHaveBeenCalledWith(500);
|
||||
expect(collect).not.toHaveBeenCalled();
|
||||
expect(result.total).toBe(1);
|
||||
expect(result.items[0]).toMatchObject({
|
||||
_id: "users:owner",
|
||||
handle: "luke",
|
||||
displayName: "Luke",
|
||||
});
|
||||
});
|
||||
|
||||
it("clamps large limit and search scan size", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:admin",
|
||||
@@ -785,6 +1204,45 @@ describe("users.list", () => {
|
||||
});
|
||||
});
|
||||
|
||||
it("includes an exact publisher-handle match even when the linked user is banned", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:admin",
|
||||
user: { _id: "users:admin", role: "admin" },
|
||||
} as never);
|
||||
const users = [
|
||||
{
|
||||
_id: "users:1",
|
||||
_creationTime: 3,
|
||||
handle: "different-login",
|
||||
displayName: "ClawGrid",
|
||||
deletedAt: 123,
|
||||
role: "user",
|
||||
},
|
||||
{ _id: "users:2", _creationTime: 2, handle: "alice", role: "user" },
|
||||
];
|
||||
const { ctx } = makeListCtx(users, {
|
||||
publishersByHandle: {
|
||||
clawgrid: {
|
||||
_id: "publishers:clawgrid",
|
||||
handle: "clawgrid",
|
||||
kind: "user",
|
||||
linkedUserId: "users:1",
|
||||
},
|
||||
},
|
||||
usersById: {
|
||||
"users:1": users[0]!,
|
||||
},
|
||||
});
|
||||
const listHandler = (
|
||||
list as unknown as { _handler: (ctx: unknown, args: unknown) => Promise<unknown> }
|
||||
)._handler;
|
||||
|
||||
await expect(listHandler(ctx, { limit: 10, search: "clawgrid" })).resolves.toMatchObject({
|
||||
total: 1,
|
||||
items: [{ _id: "users:1", deletedAt: 123 }],
|
||||
});
|
||||
});
|
||||
|
||||
it("treats whitespace search as empty search", async () => {
|
||||
vi.mocked(requireUser).mockResolvedValue({
|
||||
userId: "users:admin",
|
||||
@@ -892,6 +1350,127 @@ describe("users.searchInternal", () => {
|
||||
]);
|
||||
});
|
||||
|
||||
it("includes an exact personal publisher handle match in admin search", async () => {
|
||||
const users = [
|
||||
{ _id: "users:1", _creationTime: 2, handle: "alice", name: "alice", role: "user" },
|
||||
];
|
||||
const { ctx, get } = makeListCtx(users, {
|
||||
publishersByHandle: {
|
||||
lmlukef: {
|
||||
_id: "publishers:lmlukef",
|
||||
kind: "user",
|
||||
handle: "lmlukef",
|
||||
linkedUserId: "users:owner",
|
||||
},
|
||||
},
|
||||
usersById: {
|
||||
"users:owner": {
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "luke",
|
||||
name: "different-gh-login",
|
||||
displayName: "Luke",
|
||||
role: "user",
|
||||
},
|
||||
},
|
||||
});
|
||||
const handler = (
|
||||
searchInternal as unknown as { _handler: (ctx: unknown, args: unknown) => Promise<unknown> }
|
||||
)._handler;
|
||||
get.mockImplementation(async (id: string) => {
|
||||
if (id === "users:admin") return { _id: "users:admin", role: "admin" };
|
||||
if (id === "users:owner") {
|
||||
return {
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "luke",
|
||||
name: "different-gh-login",
|
||||
displayName: "Luke",
|
||||
role: "user",
|
||||
};
|
||||
}
|
||||
return null;
|
||||
});
|
||||
|
||||
const result = (await handler(ctx, {
|
||||
actorUserId: "users:admin",
|
||||
query: "lmLukeF",
|
||||
limit: 25,
|
||||
})) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
total: number;
|
||||
};
|
||||
|
||||
expect(result.total).toBe(1);
|
||||
expect(result.items[0]).toEqual({
|
||||
userId: "users:owner",
|
||||
handle: "luke",
|
||||
displayName: "Luke",
|
||||
name: "different-gh-login",
|
||||
role: "user",
|
||||
});
|
||||
});
|
||||
|
||||
it("does not double-count total when the fallback user already matched off-page", async () => {
|
||||
const users = [
|
||||
{
|
||||
_id: "users:1",
|
||||
_creationTime: 3,
|
||||
handle: "lmquery-top",
|
||||
name: "lmquery-top",
|
||||
role: "user",
|
||||
},
|
||||
{
|
||||
_id: "users:2",
|
||||
_creationTime: 2,
|
||||
handle: "lmquery-mid",
|
||||
name: "lmquery-mid",
|
||||
role: "user",
|
||||
},
|
||||
{
|
||||
_id: "users:owner",
|
||||
_creationTime: 1,
|
||||
handle: "owner-lmquery",
|
||||
name: "owner-lmquery",
|
||||
displayName: "Owner Lmquery",
|
||||
role: "user",
|
||||
},
|
||||
];
|
||||
const { ctx, get } = makeListCtx(users, {
|
||||
publishersByHandle: {
|
||||
lmquery: {
|
||||
_id: "publishers:lmquery",
|
||||
kind: "user",
|
||||
handle: "lmquery",
|
||||
linkedUserId: "users:owner",
|
||||
},
|
||||
},
|
||||
usersById: {
|
||||
"users:owner": users[2] as Record<string, unknown>,
|
||||
},
|
||||
});
|
||||
const handler = (
|
||||
searchInternal as unknown as { _handler: (ctx: unknown, args: unknown) => Promise<unknown> }
|
||||
)._handler;
|
||||
get.mockImplementation(async (id: string) => {
|
||||
if (id === "users:admin") return { _id: "users:admin", role: "admin" };
|
||||
if (id === "users:owner") return users[2] as Record<string, unknown>;
|
||||
return null;
|
||||
});
|
||||
|
||||
const result = (await handler(ctx, {
|
||||
actorUserId: "users:admin",
|
||||
query: "lmquery",
|
||||
limit: 2,
|
||||
})) as {
|
||||
items: Array<Record<string, unknown>>;
|
||||
total: number;
|
||||
};
|
||||
|
||||
expect(result.total).toBe(3);
|
||||
expect(result.items.map((item) => item.userId)).toEqual(["users:owner", "users:1"]);
|
||||
});
|
||||
|
||||
it("rejects deactivated actors", async () => {
|
||||
const { ctx, get } = makeListCtx([]);
|
||||
const handler = (
|
||||
@@ -914,7 +1493,7 @@ describe("users.searchInternal", () => {
|
||||
);
|
||||
});
|
||||
|
||||
it("clamps limit for empty query and uses non-search path", async () => {
|
||||
it("still caps empty-query listing and uses non-search path", async () => {
|
||||
const users = Array.from({ length: 400 }, (_value, index) => ({
|
||||
_id: `users:${index}`,
|
||||
_creationTime: 1_000 - index,
|
||||
|
||||
+123
-46
@@ -1,12 +1,16 @@
|
||||
import { getAuthUserId } from "@convex-dev/auth/server";
|
||||
import { v } from "convex/values";
|
||||
import { internal } from "./_generated/api";
|
||||
import type { Doc, Id } from "./_generated/dataModel";
|
||||
import type { ActionCtx, MutationCtx } from "./_generated/server";
|
||||
import type { ActionCtx, MutationCtx, QueryCtx } from "./_generated/server";
|
||||
import { internalAction, internalMutation, internalQuery, mutation, query } from "./functions";
|
||||
import { assertAdmin, assertModerator, requireUser } from "./lib/access";
|
||||
import { assertAdmin, assertModerator, getOptionalActiveAuthUserId, requireUser } from "./lib/access";
|
||||
import { syncGitHubProfile } from "./lib/githubAccount";
|
||||
import { ensurePersonalPublisherForUser, getPublisherByHandle } from "./lib/publishers";
|
||||
import {
|
||||
ensurePersonalPublisherForUser,
|
||||
getActiveUserByHandleOrPersonalPublisher,
|
||||
getPublisherByHandle,
|
||||
getUserByHandleOrPersonalPublisher,
|
||||
} from "./lib/publishers";
|
||||
import { toPublicUser } from "./lib/public";
|
||||
import {
|
||||
getLatestActiveReservedHandle,
|
||||
@@ -36,12 +40,7 @@ export const getByIdInternal = internalQuery({
|
||||
export const getByHandleInternal = internalQuery({
|
||||
args: { handle: v.string() },
|
||||
handler: async (ctx, args) => {
|
||||
const normalizedHandle = normalizeReservedHandle(args.handle);
|
||||
if (!normalizedHandle) return null;
|
||||
return await ctx.db
|
||||
.query("users")
|
||||
.withIndex("handle", (q) => q.eq("handle", normalizedHandle))
|
||||
.unique();
|
||||
return await getUserByHandleOrPersonalPublisher(ctx, args.handle);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -57,15 +56,28 @@ export const searchInternal = internalQuery({
|
||||
assertAdmin(actor);
|
||||
|
||||
const limit = clampInt(args.limit ?? 20, 1, MAX_USER_LIST_LIMIT);
|
||||
const result = await queryUsersForAdminList(ctx, { limit, search: args.query });
|
||||
const items = result.items.map((user) => ({
|
||||
const exactHandleUser = args.query
|
||||
? await getUserByHandleOrPersonalPublisher(ctx, args.query)
|
||||
: null;
|
||||
const result = await queryUsersForAdminList(ctx, {
|
||||
limit,
|
||||
search: args.query,
|
||||
exactUserId: exactHandleUser?._id,
|
||||
});
|
||||
const dedupedUsers = exactHandleUser
|
||||
? [exactHandleUser, ...result.items.filter((user) => user._id !== exactHandleUser._id)]
|
||||
: result.items;
|
||||
const total = exactHandleUser
|
||||
? result.total + (result.containsExactUser ? 0 : 1)
|
||||
: result.total;
|
||||
const items = dedupedUsers.slice(0, limit).map((user) => ({
|
||||
userId: user._id,
|
||||
handle: user.handle ?? null,
|
||||
displayName: user.displayName ?? null,
|
||||
name: user.name ?? null,
|
||||
role: user.role ?? null,
|
||||
}));
|
||||
return { items, total: result.total };
|
||||
return { items, total };
|
||||
},
|
||||
});
|
||||
|
||||
@@ -166,17 +178,9 @@ export const syncGitHubProfileAction = internalAction({
|
||||
export const me = query({
|
||||
args: {},
|
||||
handler: async (ctx) => {
|
||||
let userId: Awaited<ReturnType<typeof getAuthUserId>>;
|
||||
try {
|
||||
userId = await getAuthUserId(ctx);
|
||||
} catch {
|
||||
// Public pages should treat broken/stale auth as anonymous instead of crashing SSR.
|
||||
return null;
|
||||
}
|
||||
const userId = await getOptionalActiveAuthUserId(ctx);
|
||||
if (!userId) return null;
|
||||
const user = await ctx.db.get(userId);
|
||||
if (!user || user.deletedAt || user.deactivatedAt) return null;
|
||||
return user;
|
||||
return await ctx.db.get(userId);
|
||||
},
|
||||
});
|
||||
|
||||
@@ -235,6 +239,9 @@ async function computeEnsureUpdates(ctx: MutationCtx, user: Doc<"users">) {
|
||||
const updates: Record<string, unknown> = {};
|
||||
|
||||
const existingHandle = normalizeHandle(user.handle);
|
||||
const existingHandleClaimable = existingHandle
|
||||
? await canUserClaimHandle(ctx, existingHandle, user._id)
|
||||
: false;
|
||||
const githubLogin = normalizeHandle(user.name);
|
||||
const requestedHandle = deriveHandle({
|
||||
existingHandle,
|
||||
@@ -245,15 +252,20 @@ async function computeEnsureUpdates(ctx: MutationCtx, user: Doc<"users">) {
|
||||
requestedHandle && (await canUserClaimHandle(ctx, requestedHandle, user._id))
|
||||
? requestedHandle
|
||||
: undefined;
|
||||
if (!derivedHandle && !existingHandle) {
|
||||
const emailFallback = !requestedHandle && user.email ? user.email.split("@")[0]?.trim() : user.email?.split("@")[0]?.trim();
|
||||
if (!derivedHandle && (!existingHandle || !existingHandleClaimable)) {
|
||||
const emailFallback = normalizeHandle(user.email?.split("@")[0]);
|
||||
const emailFallbackHandle =
|
||||
emailFallback && emailFallback !== requestedHandle
|
||||
? await resolveAvailableHandle(ctx, emailFallback, user._id)
|
||||
: undefined;
|
||||
derivedHandle =
|
||||
(emailFallback &&
|
||||
emailFallback !== requestedHandle &&
|
||||
(await resolveAvailableHandle(ctx, emailFallback, user._id))) ||
|
||||
(await resolveAvailableHandle(ctx, requestedHandle, user._id));
|
||||
(await resolveAvailableHandle(
|
||||
ctx,
|
||||
requestedHandle ?? existingHandle ?? githubLogin ?? emailFallback,
|
||||
user._id,
|
||||
)) ?? emailFallbackHandle;
|
||||
}
|
||||
const baseHandle = derivedHandle ?? existingHandle;
|
||||
const baseHandle = derivedHandle ?? (existingHandleClaimable ? existingHandle : undefined);
|
||||
|
||||
if (derivedHandle && existingHandle !== derivedHandle) {
|
||||
updates.handle = derivedHandle;
|
||||
@@ -353,7 +365,41 @@ export const list = query({
|
||||
const { user } = await requireUser(ctx);
|
||||
assertAdmin(user);
|
||||
const limit = clampInt(args.limit ?? 50, 1, MAX_USER_LIST_LIMIT);
|
||||
return queryUsersForAdminList(ctx, { limit, search: args.search });
|
||||
const exactHandleUser = args.search
|
||||
? await getUserByHandleOrPersonalPublisher(ctx, args.search)
|
||||
: null;
|
||||
const result = await queryUsersForAdminList(ctx, {
|
||||
limit,
|
||||
search: args.search,
|
||||
exactUserId: exactHandleUser?._id,
|
||||
});
|
||||
const dedupedUsers = exactHandleUser
|
||||
? [exactHandleUser, ...result.items.filter((entry) => entry._id !== exactHandleUser._id)]
|
||||
: result.items;
|
||||
const total = exactHandleUser
|
||||
? result.total + (result.containsExactUser ? 0 : 1)
|
||||
: result.total;
|
||||
return {
|
||||
items: dedupedUsers.slice(0, limit),
|
||||
total,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
export const listPublic = query({
|
||||
args: { limit: v.optional(v.number()), search: v.optional(v.string()) },
|
||||
handler: async (ctx, args) => {
|
||||
const limit = clampInt(args.limit ?? 40, 1, 100);
|
||||
const result = await queryUsersForPublicList(ctx, {
|
||||
limit,
|
||||
search: args.search,
|
||||
});
|
||||
return {
|
||||
items: result.items
|
||||
.map((user) => toPublicUser(user))
|
||||
.filter((user): user is NonNullable<ReturnType<typeof toPublicUser>> => Boolean(user)),
|
||||
total: result.total,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
@@ -367,26 +413,47 @@ function computeUserSearchScanLimit(limit: number) {
|
||||
}
|
||||
|
||||
async function queryUsersForAdminList(
|
||||
ctx: {
|
||||
db: {
|
||||
query: (table: "users") => {
|
||||
order: (order: "desc") => { take: (n: number) => Promise<Doc<"users">[]> };
|
||||
};
|
||||
};
|
||||
},
|
||||
args: { limit: number; search?: string },
|
||||
ctx: Pick<QueryCtx, "db">,
|
||||
args: { limit: number; search?: string; exactUserId?: Id<"users"> },
|
||||
) {
|
||||
const normalizedSearch = normalizeSearchQuery(args.search);
|
||||
const orderedUsers = ctx.db.query("users").order("desc");
|
||||
|
||||
if (!normalizedSearch) {
|
||||
const items = await orderedUsers.take(args.limit);
|
||||
return { items, total: items.length };
|
||||
return { items, total: items.length, containsExactUser: false };
|
||||
}
|
||||
|
||||
const scannedUsers = await orderedUsers.take(computeUserSearchScanLimit(args.limit));
|
||||
const result = buildUserSearchResults(scannedUsers, normalizedSearch);
|
||||
return { items: result.items.slice(0, args.limit), total: result.total };
|
||||
return {
|
||||
items: result.items.slice(0, args.limit),
|
||||
total: result.total,
|
||||
containsExactUser: args.exactUserId
|
||||
? result.items.some((user) => user._id === args.exactUserId)
|
||||
: false,
|
||||
};
|
||||
}
|
||||
|
||||
async function queryUsersForPublicList(
|
||||
ctx: Pick<QueryCtx, "db">,
|
||||
args: { limit: number; search?: string },
|
||||
) {
|
||||
const normalizedSearch = normalizeSearchQuery(args.search);
|
||||
const scanLimit = normalizedSearch
|
||||
? computeUserSearchScanLimit(args.limit)
|
||||
: clampInt(args.limit * 6, args.limit, MAX_USER_SEARCH_SCAN);
|
||||
const scannedUsers = await ctx.db
|
||||
.query("users")
|
||||
.withIndex("by_active_handle", (q) => q.eq("deletedAt", undefined).eq("deactivatedAt", undefined))
|
||||
.order("desc")
|
||||
.take(scanLimit);
|
||||
const activeUsers = scannedUsers.filter((user) => Boolean(user.handle));
|
||||
const result = buildUserSearchResults(activeUsers, normalizedSearch);
|
||||
return {
|
||||
items: result.items.slice(0, args.limit),
|
||||
total: result.total,
|
||||
};
|
||||
}
|
||||
|
||||
function clampInt(value: number, min: number, max: number) {
|
||||
@@ -396,11 +463,21 @@ function clampInt(value: number, min: number, max: number) {
|
||||
export const getByHandle = query({
|
||||
args: { handle: v.string() },
|
||||
handler: async (ctx, args) => {
|
||||
const user = await ctx.db
|
||||
.query("users")
|
||||
.withIndex("handle", (q) => q.eq("handle", args.handle))
|
||||
.unique();
|
||||
return toPublicUser(user);
|
||||
return toPublicUser(await getActiveUserByHandleOrPersonalPublisher(ctx, args.handle));
|
||||
},
|
||||
});
|
||||
|
||||
/** Lightweight stats for user hover tooltips. Uses the skills by_owner index. */
|
||||
export const getHoverStats = query({
|
||||
args: { userId: v.id("users") },
|
||||
handler: async (ctx, args) => {
|
||||
const user = await ctx.db.get(args.userId);
|
||||
|
||||
return {
|
||||
publishedSkills: user?.publishedSkills ?? 0,
|
||||
totalStars: user?.totalStars ?? 0,
|
||||
totalDownloads: user?.totalDownloads ?? 0,
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
|
||||
+573
-2
@@ -1,5 +1,37 @@
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { __test } from "./vt";
|
||||
/* @vitest-environment node */
|
||||
|
||||
import { afterEach, describe, expect, it, vi } from "vitest";
|
||||
import { __test, pollPackageReleaseScanResults, scanPackageReleaseWithVirusTotal } from "./vt";
|
||||
|
||||
type WrappedHandler<TArgs, TResult> = {
|
||||
_handler: (ctx: unknown, args: TArgs) => Promise<TResult>;
|
||||
};
|
||||
|
||||
const scanPackageReleaseWithVirusTotalHandler = (
|
||||
scanPackageReleaseWithVirusTotal as unknown as WrappedHandler<
|
||||
{ releaseId: string; attempt?: number },
|
||||
void
|
||||
>
|
||||
)._handler;
|
||||
|
||||
const pollPackageReleaseScanResultsHandler = (
|
||||
pollPackageReleaseScanResults as unknown as WrappedHandler<
|
||||
{ releaseId: string; attempt?: number },
|
||||
void
|
||||
>
|
||||
)._handler;
|
||||
|
||||
const originalVtApiKey = process.env.VT_API_KEY;
|
||||
|
||||
afterEach(() => {
|
||||
if (originalVtApiKey === undefined) {
|
||||
delete process.env.VT_API_KEY;
|
||||
} else {
|
||||
process.env.VT_API_KEY = originalVtApiKey;
|
||||
}
|
||||
vi.restoreAllMocks();
|
||||
vi.unstubAllGlobals();
|
||||
});
|
||||
|
||||
describe("vt activation fallback", () => {
|
||||
it("activates only VT-pending hidden skills", () => {
|
||||
@@ -100,3 +132,542 @@ describe("vt AV engine fallback verdicts", () => {
|
||||
).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe("package VT retries", () => {
|
||||
it("retries package scan when release files are not readable yet", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
}),
|
||||
runMutation: vi.fn(async () => null),
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(async () => null),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo", attempt: 2 },
|
||||
);
|
||||
|
||||
expect(scheduler.runAfter).toHaveBeenCalledWith(5 * 60 * 1000, expect.anything(), {
|
||||
releaseId: "packageReleases:demo",
|
||||
attempt: 3,
|
||||
});
|
||||
});
|
||||
|
||||
it("retries package upload when VT upload fails", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce(new Response("", { status: 404 }))
|
||||
.mockResolvedValueOnce(new Response("rate limited", { status: 429 }));
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(
|
||||
async () => new Blob(['{"name":"demo-plugin"}'], { type: "application/json" }),
|
||||
),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo" },
|
||||
);
|
||||
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
releaseId: "packageReleases:demo",
|
||||
sha256hash: expect.any(String),
|
||||
}),
|
||||
);
|
||||
expect(scheduler.runAfter).toHaveBeenCalledWith(5 * 60 * 1000, expect.anything(), {
|
||||
releaseId: "packageReleases:demo",
|
||||
attempt: 2,
|
||||
});
|
||||
});
|
||||
|
||||
it("uses existing AV engine verdicts for packages without re-uploading", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi.fn().mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 1,
|
||||
harmless: 10,
|
||||
undetected: 40,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "clean" },
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
family: "code-plugin",
|
||||
isOfficial: true,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(
|
||||
async () => new Blob(['{"name":"demo-plugin"}'], { type: "application/json" }),
|
||||
),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo" },
|
||||
);
|
||||
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
releaseId: "packageReleases:demo",
|
||||
vtAnalysis: expect.objectContaining({ status: "suspicious", source: "engines" }),
|
||||
}),
|
||||
);
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(scheduler.runAfter).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not promote official source-linked packages with suspicious static scans via fallback", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({ data: { id: "analysis-123" } }),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "suspicious" },
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
family: "code-plugin",
|
||||
isOfficial: true,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(
|
||||
async () => new Blob(['{"name":"demo-plugin"}'], { type: "application/json" }),
|
||||
),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo" },
|
||||
);
|
||||
|
||||
expect(runMutation.mock.calls.length).toBe(1);
|
||||
const mutationCalls = runMutation.mock.calls as unknown as Array<
|
||||
[unknown, Record<string, unknown>]
|
||||
>;
|
||||
expect(mutationCalls.some(([, payload]) => "vtAnalysis" in payload)).toBe(false);
|
||||
expect(fetchMock.mock.calls.length).toBe(2);
|
||||
expect(scheduler.runAfter.mock.calls.length).toBe(1);
|
||||
});
|
||||
|
||||
it("promotes official source-linked packages with clean static scans via fallback", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi.fn().mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "clean" },
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
family: "code-plugin",
|
||||
isOfficial: true,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(
|
||||
async () => new Blob(['{"name":"demo-plugin"}'], { type: "application/json" }),
|
||||
),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo" },
|
||||
);
|
||||
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
releaseId: "packageReleases:demo",
|
||||
vtAnalysis: expect.objectContaining({
|
||||
status: "clean",
|
||||
source: "engines-undetected-fallback",
|
||||
verdict: "undetected-only-fallback",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(scheduler.runAfter).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("promotes community source-linked packages with undetected-only VT stats via fallback", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi.fn().mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await scanPackageReleaseWithVirusTotalHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "clean" },
|
||||
files: [{ path: "package.json", storageId: "storage:pkg" }],
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
name: "demo-plugin",
|
||||
family: "code-plugin",
|
||||
isOfficial: false,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
storage: {
|
||||
get: vi.fn(
|
||||
async () => new Blob(['{"name":"demo-plugin"}'], { type: "application/json" }),
|
||||
),
|
||||
},
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo" },
|
||||
);
|
||||
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
releaseId: "packageReleases:demo",
|
||||
vtAnalysis: expect.objectContaining({
|
||||
status: "clean",
|
||||
source: "engines-undetected-fallback",
|
||||
verdict: "undetected-only-fallback",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(fetchMock).toHaveBeenCalledTimes(1);
|
||||
expect(scheduler.runAfter).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("retries package poll when VT lookup throws", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
vi.stubGlobal("fetch", vi.fn().mockRejectedValue(new Error("network error")));
|
||||
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await pollPackageReleaseScanResultsHandler(
|
||||
{
|
||||
runQuery: vi.fn().mockResolvedValue({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
sha256hash: "abc123",
|
||||
}),
|
||||
runMutation: vi.fn(async () => null),
|
||||
scheduler,
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo", attempt: 3 },
|
||||
);
|
||||
|
||||
expect(scheduler.runAfter).toHaveBeenCalledWith(5 * 60 * 1000, expect.anything(), {
|
||||
releaseId: "packageReleases:demo",
|
||||
attempt: 4,
|
||||
});
|
||||
});
|
||||
|
||||
it("does not apply undetected-only fallback during package polling when static scan is suspicious", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
}),
|
||||
);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await pollPackageReleaseScanResultsHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
sha256hash: "abc123",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "suspicious" },
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
family: "code-plugin",
|
||||
isOfficial: true,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo", attempt: 3 },
|
||||
);
|
||||
|
||||
expect(runMutation).not.toHaveBeenCalled();
|
||||
expect(scheduler.runAfter).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it("applies the same undetected-only fallback during community package polling", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
vi.stubGlobal(
|
||||
"fetch",
|
||||
vi.fn().mockResolvedValue({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
}),
|
||||
);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await pollPackageReleaseScanResultsHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
sha256hash: "abc123",
|
||||
verification: { tier: "source-linked" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "clean" },
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
family: "code-plugin",
|
||||
isOfficial: false,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo", attempt: 3 },
|
||||
);
|
||||
|
||||
expect(runMutation).toHaveBeenCalledWith(
|
||||
expect.anything(),
|
||||
expect.objectContaining({
|
||||
releaseId: "packageReleases:demo",
|
||||
vtAnalysis: expect.objectContaining({
|
||||
status: "clean",
|
||||
source: "engines-undetected-fallback",
|
||||
verdict: "undetected-only-fallback",
|
||||
}),
|
||||
}),
|
||||
);
|
||||
expect(scheduler.runAfter).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it("does not promote undetected-only community packages without trusted verification", async () => {
|
||||
process.env.VT_API_KEY = "test-key";
|
||||
const fetchMock = vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({
|
||||
data: {
|
||||
attributes: {
|
||||
last_analysis_stats: {
|
||||
malicious: 0,
|
||||
suspicious: 0,
|
||||
harmless: 0,
|
||||
undetected: 66,
|
||||
},
|
||||
},
|
||||
},
|
||||
}),
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
ok: true,
|
||||
json: async () => ({}),
|
||||
});
|
||||
vi.stubGlobal("fetch", fetchMock);
|
||||
|
||||
const runMutation = vi.fn(async () => null);
|
||||
const scheduler = { runAfter: vi.fn(async () => null) };
|
||||
await pollPackageReleaseScanResultsHandler(
|
||||
{
|
||||
runQuery: vi
|
||||
.fn()
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packageReleases:demo",
|
||||
packageId: "packages:demo",
|
||||
version: "1.0.0",
|
||||
sha256hash: "abc123",
|
||||
verification: { tier: "artifact-only" },
|
||||
llmAnalysis: { status: "clean" },
|
||||
staticScan: { status: "clean" },
|
||||
})
|
||||
.mockResolvedValueOnce({
|
||||
_id: "packages:demo",
|
||||
family: "code-plugin",
|
||||
isOfficial: false,
|
||||
}),
|
||||
runMutation,
|
||||
scheduler,
|
||||
} as never,
|
||||
{ releaseId: "packageReleases:demo", attempt: 3 },
|
||||
);
|
||||
|
||||
expect(runMutation).not.toHaveBeenCalled();
|
||||
expect(fetchMock).toHaveBeenCalledTimes(2);
|
||||
expect(scheduler.runAfter).toHaveBeenCalledWith(5 * 60 * 1000, expect.anything(), {
|
||||
releaseId: "packageReleases:demo",
|
||||
attempt: 4,
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
+159
-52
@@ -12,6 +12,7 @@ const internalRefs = internal as unknown as {
|
||||
updateReleaseScanResultsInternal: unknown;
|
||||
};
|
||||
vt: {
|
||||
scanPackageReleaseWithVirusTotal: unknown;
|
||||
pollPackageReleaseScanResults: unknown;
|
||||
};
|
||||
};
|
||||
@@ -159,6 +160,70 @@ type VTFileResponse = {
|
||||
};
|
||||
|
||||
type VTAnalysisStats = NonNullable<VTFileResponse["data"]["attributes"]["last_analysis_stats"]>;
|
||||
type PackageReleaseScanDoc = Pick<
|
||||
Doc<"packageReleases">,
|
||||
"verification" | "llmAnalysis" | "staticScan"
|
||||
>;
|
||||
type PackageScanDoc = Pick<Doc<"packages">, "family" | "isOfficial">;
|
||||
|
||||
function buildPackageUndetectedFallbackAnalysis(
|
||||
release: PackageReleaseScanDoc,
|
||||
pkg: PackageScanDoc,
|
||||
stats?: VTAnalysisStats,
|
||||
) {
|
||||
if (!stats) return null;
|
||||
if (pkg.family === "skill") return null;
|
||||
|
||||
const tier = release.verification?.tier;
|
||||
if (tier !== "source-linked" && tier !== "provenance-verified" && tier !== "rebuild-verified") {
|
||||
return null;
|
||||
}
|
||||
if (release.llmAnalysis?.status !== "clean") return null;
|
||||
if (!release.staticScan || release.staticScan.status !== "clean") return null;
|
||||
if (stats.malicious !== 0 || stats.suspicious !== 0) return null;
|
||||
if ((stats.harmless ?? 0) <= 0 && (stats.undetected ?? 0) <= 0) return null;
|
||||
|
||||
return {
|
||||
status: "clean",
|
||||
verdict: "undetected-only-fallback",
|
||||
analysis:
|
||||
"VirusTotal reported no malicious or suspicious engine hits. ClawHub promoted this source-linked package after clean LLM and clean static scans.",
|
||||
source: "engines-undetected-fallback",
|
||||
checkedAt: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
function buildPackageScanAnalysisFromVtResult(
|
||||
release: PackageReleaseScanDoc,
|
||||
pkg: PackageScanDoc,
|
||||
vtResult: VTFileResponse,
|
||||
) {
|
||||
const aiResult = vtResult.data.attributes.crowdsourced_ai_results?.find(
|
||||
(r) => r.category === "code_insight",
|
||||
);
|
||||
if (aiResult) {
|
||||
const verdict = normalizeVerdict(aiResult.verdict);
|
||||
return {
|
||||
status: verdictToStatus(verdict),
|
||||
verdict: aiResult.verdict,
|
||||
analysis: aiResult.analysis,
|
||||
source: aiResult.source,
|
||||
checkedAt: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
const stats = vtResult.data.attributes.last_analysis_stats;
|
||||
const status = statusFromAvStats(stats);
|
||||
if (status) {
|
||||
return {
|
||||
status,
|
||||
source: "engines",
|
||||
checkedAt: Date.now(),
|
||||
};
|
||||
}
|
||||
|
||||
return buildPackageUndetectedFallbackAnalysis(release, pkg, stats);
|
||||
}
|
||||
|
||||
type ScanQueueHealth = {
|
||||
queueSize: number;
|
||||
@@ -526,6 +591,7 @@ const PACKAGE_SCAN_MAX_ATTEMPTS = 10;
|
||||
export const scanPackageReleaseWithVirusTotal = internalAction({
|
||||
args: {
|
||||
releaseId: v.id("packageReleases"),
|
||||
attempt: v.optional(v.number()),
|
||||
},
|
||||
handler: async (ctx, args) => {
|
||||
const apiKey = process.env.VT_API_KEY;
|
||||
@@ -550,17 +616,35 @@ export const scanPackageReleaseWithVirusTotal = internalAction({
|
||||
return;
|
||||
}
|
||||
|
||||
const attempt = args.attempt ?? 1;
|
||||
const entries: Array<{ path: string; bytes: Uint8Array }> = [];
|
||||
let missingFiles = 0;
|
||||
for (const file of release.files) {
|
||||
const content = await ctx.storage.get(file.storageId);
|
||||
if (!content) continue;
|
||||
if (!content) {
|
||||
missingFiles += 1;
|
||||
continue;
|
||||
}
|
||||
entries.push({
|
||||
path: file.path,
|
||||
bytes: new Uint8Array(await content.arrayBuffer()),
|
||||
});
|
||||
}
|
||||
if (entries.length === 0) {
|
||||
console.warn(`[vt:package] No files found for release ${args.releaseId}, skipping scan`);
|
||||
if (entries.length === 0 || missingFiles > 0) {
|
||||
console.warn(
|
||||
`[vt:package] Release ${args.releaseId} missing ${missingFiles}/${release.files.length} files, retrying`,
|
||||
);
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.scanPackageReleaseWithVirusTotal,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
@@ -577,21 +661,14 @@ export const scanPackageReleaseWithVirusTotal = internalAction({
|
||||
|
||||
try {
|
||||
const existingFile = await checkExistingFile(apiKey, sha256hash);
|
||||
const aiResult = existingFile?.data.attributes.crowdsourced_ai_results?.find(
|
||||
(r) => r.category === "code_insight",
|
||||
);
|
||||
const vtAnalysis = existingFile
|
||||
? buildPackageScanAnalysisFromVtResult(release, pkg, existingFile)
|
||||
: null;
|
||||
|
||||
if (aiResult) {
|
||||
const verdict = normalizeVerdict(aiResult.verdict);
|
||||
if (vtAnalysis) {
|
||||
await runMutationRef(ctx, internalRefs.packages.updateReleaseScanResultsInternal, {
|
||||
releaseId: args.releaseId,
|
||||
vtAnalysis: {
|
||||
status: verdictToStatus(verdict),
|
||||
verdict: aiResult.verdict,
|
||||
analysis: aiResult.analysis,
|
||||
source: aiResult.source,
|
||||
checkedAt: Date.now(),
|
||||
},
|
||||
vtAnalysis,
|
||||
});
|
||||
return;
|
||||
}
|
||||
@@ -613,19 +690,46 @@ export const scanPackageReleaseWithVirusTotal = internalAction({
|
||||
if (!response.ok) {
|
||||
const error = await response.text();
|
||||
console.error("[vt:package] VirusTotal upload error:", error);
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.scanPackageReleaseWithVirusTotal,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
await runAfterRef(ctx, PACKAGE_SCAN_RETRY_DELAY_MS, internalRefs.vt.pollPackageReleaseScanResults, {
|
||||
releaseId: args.releaseId,
|
||||
attempt: 1,
|
||||
});
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.pollPackageReleaseScanResults,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: 1,
|
||||
},
|
||||
);
|
||||
|
||||
console.log(
|
||||
`[vt:package] Uploaded ${pkg.name}@${release.version} for scanning (${sha256hash})`,
|
||||
);
|
||||
} catch (error) {
|
||||
console.error("[vt:package] Failed to upload to VirusTotal:", error);
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.scanPackageReleaseWithVirusTotal,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
},
|
||||
});
|
||||
@@ -643,60 +747,63 @@ export const pollPackageReleaseScanResults = internalAction({
|
||||
releaseId: args.releaseId,
|
||||
})) as Doc<"packageReleases"> | null;
|
||||
if (!release || release.softDeletedAt || !release.sha256hash) return;
|
||||
const pkg = (await runQueryRef(ctx, internalRefs.packages.getPackageByIdInternal, {
|
||||
packageId: release.packageId,
|
||||
})) as Doc<"packages"> | null;
|
||||
if (!pkg || pkg.softDeletedAt) return;
|
||||
|
||||
const attempt = args.attempt ?? 1;
|
||||
try {
|
||||
const vtResult = await checkExistingFile(apiKey, release.sha256hash);
|
||||
if (!vtResult) {
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(ctx, PACKAGE_SCAN_RETRY_DELAY_MS, internalRefs.vt.pollPackageReleaseScanResults, {
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
});
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.pollPackageReleaseScanResults,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
const aiResult = vtResult.data.attributes.crowdsourced_ai_results?.find(
|
||||
(r) => r.category === "code_insight",
|
||||
);
|
||||
if (aiResult) {
|
||||
const verdict = normalizeVerdict(aiResult.verdict);
|
||||
const vtAnalysis = buildPackageScanAnalysisFromVtResult(release, pkg, vtResult);
|
||||
if (vtAnalysis) {
|
||||
await runMutationRef(ctx, internalRefs.packages.updateReleaseScanResultsInternal, {
|
||||
releaseId: args.releaseId,
|
||||
vtAnalysis: {
|
||||
status: verdictToStatus(verdict),
|
||||
verdict: aiResult.verdict,
|
||||
analysis: aiResult.analysis,
|
||||
source: aiResult.source,
|
||||
checkedAt: Date.now(),
|
||||
},
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
const status = statusFromAvStats(vtResult.data.attributes.last_analysis_stats);
|
||||
if (status) {
|
||||
await runMutationRef(ctx, internalRefs.packages.updateReleaseScanResultsInternal, {
|
||||
releaseId: args.releaseId,
|
||||
vtAnalysis: {
|
||||
status,
|
||||
source: "engines",
|
||||
checkedAt: Date.now(),
|
||||
},
|
||||
vtAnalysis,
|
||||
});
|
||||
return;
|
||||
}
|
||||
|
||||
await requestRescan(apiKey, release.sha256hash);
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(ctx, PACKAGE_SCAN_RETRY_DELAY_MS, internalRefs.vt.pollPackageReleaseScanResults, {
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
});
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.pollPackageReleaseScanResults,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error(`[vt:package] Error polling ${release.sha256hash}:`, error);
|
||||
if (attempt < PACKAGE_SCAN_MAX_ATTEMPTS) {
|
||||
await runAfterRef(
|
||||
ctx,
|
||||
PACKAGE_SCAN_RETRY_DELAY_MS,
|
||||
internalRefs.vt.pollPackageReleaseScanResults,
|
||||
{
|
||||
releaseId: args.releaseId,
|
||||
attempt: attempt + 1,
|
||||
},
|
||||
);
|
||||
}
|
||||
}
|
||||
},
|
||||
});
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
summary: "Marketplace policy: what ClawHub will not allow."
|
||||
read_when:
|
||||
- Reviewing uploads for abuse or policy violations
|
||||
- Writing moderation docs or reviewer runbooks
|
||||
- Deciding whether a skill should be hidden or a user banned
|
||||
---
|
||||
|
||||
# Acceptable Usage
|
||||
|
||||
This page describes the kinds of skills and content ClawHub is not okay with.
|
||||
|
||||
These rules are intentionally practical. We care most about end-to-end abuse workflows, not just isolated keywords. If a skill is built to evade defenses, abuse platforms, scam people, invade privacy, or enable non-consensual behavior, it does not belong on ClawHub.
|
||||
|
||||
## Not okay
|
||||
|
||||
- Security-bypass or unauthorized-access workflows.
|
||||
- Examples: auth bypass, account takeover, CAPTCHA bypass, Cloudflare or anti-bot evasion, rate-limit bypass, stealth scraping designed to defeat protections, live call or agent takeover, reusable session theft, auto-approving pairing flows for unapproved users.
|
||||
|
||||
- Platform abuse and ban evasion.
|
||||
- Examples: stealth accounts after bans, account warming/farming, fake engagement, karma or follower cultivation, multi-account automation, mass posting, spam bots, marketplace or social automation built to avoid detection.
|
||||
|
||||
- Fraud, scams, and deceptive financial workflows.
|
||||
- Examples: fake certificates, fake invoices, deceptive payment flows, scam outreach, fake social proof, tools that enable spending or charging without clear human approval and transparent controls, or synthetic-identity workflows built to create accounts for fraud.
|
||||
|
||||
- Privacy-invasive scraping, enrichment, or surveillance.
|
||||
- Examples: scraping contact details at scale for spam, doxxing, stalking, lead extraction paired with unsolicited outreach, covert monitoring, face search or biometric matching used without clear consent, or buying, publishing, downloading, or operationalizing leaked data or breach dumps.
|
||||
|
||||
- Non-consensual impersonation or deceptive identity manipulation.
|
||||
- Examples: face swap, digital twins, fake personas, cloned influencers, or other identity-manipulation tooling used to impersonate or mislead.
|
||||
|
||||
- Explicit sexual content and safety-disabled adult generation.
|
||||
- Examples: NSFW image/video/content generation, adult-content wrappers around third-party APIs, or skills whose primary purpose is explicit sexual content.
|
||||
|
||||
- Hidden, unsafe, or misleading execution requirements.
|
||||
- Examples: obfuscated install commands, `curl | sh`, undeclared secret requirements, undeclared private-key use, remote `npx @latest` execution without clear reviewability, misleading metadata that hides what the skill really needs to run.
|
||||
|
||||
## Recent patterns we are explicitly not okay with
|
||||
|
||||
- “Create stealth seller accounts after marketplace bans.”
|
||||
- “Modify Telegram pairing so unapproved users automatically receive pairing codes.”
|
||||
- “Cultivate Reddit/Twitter accounts with undetectable automation.”
|
||||
- “Generate professional certificates or invoices for arbitrary use.”
|
||||
- “Generate NSFW content with safety checks disabled.”
|
||||
- “Scrape leads, enrich contacts, and launch cold outreach at scale.”
|
||||
- “Buy, publish, or download leaked data or breach dumps.”
|
||||
- “Bulk-create email or social accounts with synthetic identities or CAPTCHA solving.”
|
||||
|
||||
## Notes for reviewers
|
||||
|
||||
- Context matters. The same topic can be legitimate in a narrow defensive or consent-based setting and unacceptable when packaged as an abuse workflow.
|
||||
- We should bias toward action when a skill is clearly optimized for evasion, deception, or non-consensual use.
|
||||
- Repeated uploads in these categories are grounds for hiding content and banning the account.
|
||||
|
||||
## Enforcement
|
||||
|
||||
- We may hide, remove, or hard-delete violating skills.
|
||||
- We may revoke tokens, soft-delete associated content, and ban repeat or severe offenders.
|
||||
- We do not guarantee warning-first enforcement for obvious abuse.
|
||||
+2
-2
@@ -24,8 +24,8 @@ Auth-aware enforcement:
|
||||
- Authenticated requests (valid Bearer token): per user bucket.
|
||||
- Missing/invalid token falls back to IP enforcement.
|
||||
|
||||
- Read: 120/min per IP, 600/min per key
|
||||
- Write: 30/min per IP, 120/min per key
|
||||
- Read: 180/min per IP, 900/min per key
|
||||
- Write: 45/min per IP, 180/min per key
|
||||
|
||||
Headers: `X-RateLimit-Limit`, `X-RateLimit-Remaining`, `X-RateLimit-Reset`, `RateLimit-Limit`, `RateLimit-Remaining`, `RateLimit-Reset`, `Retry-After` (on 429).
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ read_when:
|
||||
|
||||
- Web app: TanStack Start (React) under `src/`.
|
||||
- Backend: Convex under `convex/` (DB, storage, actions, HTTP routes).
|
||||
- CLI: `packages/clawdhub/` (published as `clawhub`, legacy `clawdhub`).
|
||||
- CLI: `packages/clawhub/` (published as `clawhub`, legacy `clawdhub`).
|
||||
- Shared schemas/routes: `packages/schema/` (`clawhub-schema`).
|
||||
|
||||
## Data + storage
|
||||
|
||||
@@ -11,6 +11,7 @@ read_when:
|
||||
|
||||
- Convex Auth + GitHub OAuth App.
|
||||
- GitHub is the only supported login provider.
|
||||
- Disabled/banned accounts are blocked during OAuth completion and should surface a user-facing reason instead of a generic auth failure.
|
||||
- Env vars:
|
||||
- `AUTH_GITHUB_ID`
|
||||
- `AUTH_GITHUB_SECRET`
|
||||
|
||||
+66
-4
@@ -7,7 +7,7 @@ read_when:
|
||||
|
||||
# CLI
|
||||
|
||||
CLI package: `packages/clawdhub/` (published as `clawhub`, bin: `clawhub`).
|
||||
CLI package: `packages/clawhub/` (published as `clawhub`, bin: `clawhub`).
|
||||
|
||||
From this repo you can run it via the wrapper script:
|
||||
|
||||
@@ -62,6 +62,9 @@ When no proxy variable is set, behavior is unchanged (direct connections).
|
||||
Stores your API token + cached registry URL.
|
||||
|
||||
- macOS: `~/Library/Application Support/clawhub/config.json`
|
||||
- Linux/XDG: `$XDG_CONFIG_HOME/clawhub/config.json` or `~/.config/clawhub/config.json`
|
||||
- Windows: `%APPDATA%\\clawhub\\config.json`
|
||||
- Legacy fallback: if `clawhub/config.json` does not exist yet but `clawdhub/config.json` does, the CLI reuses the legacy path
|
||||
- override: `CLAWHUB_CONFIG_PATH` (legacy `CLAWDHUB_CONFIG_PATH`)
|
||||
|
||||
## Commands
|
||||
@@ -132,12 +135,13 @@ Stores your API token + cached registry URL.
|
||||
- refuses by default
|
||||
- overwrites with `--force` (or prompt, if interactive)
|
||||
|
||||
### `publish <path>`
|
||||
### `skill publish <path>`
|
||||
|
||||
- Publishes via `POST /api/v1/skills` (multipart).
|
||||
- Requires semver: `--version 1.2.3`.
|
||||
- Publishing a skill means it is released under `MIT-0` on ClawHub.
|
||||
- Published skills are free to use, modify, and redistribute without attribution.
|
||||
- Legacy alias: `publish <path>`.
|
||||
|
||||
### `delete <slug>`
|
||||
|
||||
@@ -208,11 +212,69 @@ Stores your API token + cached registry URL.
|
||||
- `--fuzzy` resolves the handle via fuzzy user search (admin only).
|
||||
- `--yes` skips confirmation.
|
||||
|
||||
### `package publish <path>`
|
||||
### `package publish <source>`
|
||||
|
||||
- Publishes a code plugin or bundle plugin via `POST /api/v1/packages`.
|
||||
- `<source>` accepts:
|
||||
- Local folder path: `./my-plugin`
|
||||
- GitHub repo: `owner/repo` or `owner/repo@ref`
|
||||
- GitHub URL: `https://github.com/owner/repo`
|
||||
- Metadata is auto-detected from `package.json`, `openclaw.plugin.json`, and `openclaw.bundle.json`.
|
||||
- For GitHub sources, source attribution is auto-populated from the repo, resolved commit, ref, and subpath.
|
||||
- For local folders, source attribution is auto-detected from local git when the origin remote points at GitHub.
|
||||
- External code plugins must declare `openclaw.compat.pluginApi` and `openclaw.build.openclawVersion` explicitly.
|
||||
Top-level `package.json.version` is not used as a fallback for publish validation.
|
||||
- `--dry-run` previews the resolved publish payload without uploading.
|
||||
- `--json` emits machine-readable output for CI.
|
||||
- `--owner <handle>` lets admins publish under a shared owner account while keeping their own token as the actor.
|
||||
- Code plugins still require `--source-repo` and `--source-commit`.
|
||||
- Existing flags (`--family`, `--name`, `--version`, `--source-repo`, `--source-commit`, `--source-ref`, `--source-path`) still work as overrides.
|
||||
- Private GitHub repos require `GITHUB_TOKEN`.
|
||||
|
||||
#### GitHub Actions
|
||||
|
||||
ClawHub also ships an official reusable workflow at
|
||||
[`/.github/workflows/package-publish.yml`](../.github/workflows/package-publish.yml)
|
||||
for plugin repos.
|
||||
|
||||
Typical caller setup:
|
||||
|
||||
```yaml
|
||||
name: Package Publish
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
workflow_dispatch:
|
||||
push:
|
||||
tags:
|
||||
- "v*"
|
||||
|
||||
jobs:
|
||||
dry-run:
|
||||
if: github.event_name == 'pull_request'
|
||||
uses: openclaw/clawhub/.github/workflows/package-publish.yml@main
|
||||
with:
|
||||
dry_run: true
|
||||
|
||||
publish:
|
||||
if: github.event_name == 'workflow_dispatch' || startsWith(github.ref, 'refs/tags/')
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
uses: openclaw/clawhub/.github/workflows/package-publish.yml@main
|
||||
with:
|
||||
dry_run: false
|
||||
secrets:
|
||||
clawhub_token: ${{ secrets.CLAWHUB_TOKEN }}
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- The reusable workflow defaults `source` to the caller repo.
|
||||
- `pull_request` should use `dry_run: true` so CI stays non-polluting.
|
||||
- Real publishes should be limited to trusted events such as `workflow_dispatch` or tag pushes.
|
||||
- Trusted publishing without a secret only works on `workflow_dispatch`; tag pushes still need `clawhub_token`.
|
||||
- Keep `clawhub_token` available for first publish, untrusted packages, or break-glass publishes.
|
||||
- The workflow uploads the JSON result as an artifact and exposes it as workflow outputs.
|
||||
|
||||
### `sync`
|
||||
|
||||
|
||||
+42
-9
@@ -25,21 +25,54 @@ bunx convex deploy
|
||||
Or use the GitHub Actions pipeline:
|
||||
|
||||
```bash
|
||||
gh workflow run deploy.yml
|
||||
gh workflow run deploy.yml --repo openclaw/clawhub --ref main
|
||||
```
|
||||
|
||||
GitHub Actions secrets required for `deploy.yml`:
|
||||
Production deploy notes:
|
||||
|
||||
- `CONVEX_DEPLOY_KEY`
|
||||
- Optional: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON` for authenticated smoke coverage
|
||||
- `deploy.yml` is manual-only (`workflow_dispatch`). Merging to `main` does not deploy.
|
||||
- The workflow must be started from `main`.
|
||||
- Deploy targets:
|
||||
- `full`: deploy Convex, verify contract, wait for the matching Vercel production deploy, then run smoke tests
|
||||
- `backend`: deploy Convex, verify contract, then run smoke tests against current production
|
||||
- `frontend`: wait for the Vercel production deploy for the selected `main` SHA, then run smoke tests
|
||||
- `frontend` does not call `vercel deploy` directly yet. It relies on the existing Vercel Git-based production deploy for that SHA.
|
||||
- The real deploy job uses the GitHub `Production` environment for deploy secrets, but it does not wait for a separate approval.
|
||||
- Required `Production` environment secret: `CONVEX_DEPLOY_KEY`.
|
||||
- Optional `Production` environment secret: `PLAYWRIGHT_AUTH_STORAGE_STATE_JSON` for authenticated smoke coverage.
|
||||
|
||||
`deploy.yml` now fails in preflight if `CONVEX_DEPLOY_KEY` is missing. Web deploy
|
||||
verification no longer depends on a separate Vercel token in GitHub Actions.
|
||||
## CLI npm release
|
||||
|
||||
The `clawhub` CLI package is released separately from the app deploy.
|
||||
Only stable releases are supported here: `vX.Y.Z`.
|
||||
|
||||
Use the GitHub Actions workflow:
|
||||
|
||||
```bash
|
||||
gh workflow run clawhub-cli-npm-release.yml \
|
||||
--repo openclaw/clawhub \
|
||||
--ref main \
|
||||
-f tag=v0.10.0 \
|
||||
-f preflight_only=true
|
||||
```
|
||||
|
||||
Then rerun the same workflow from `main` with:
|
||||
|
||||
- the same `tag`
|
||||
- `preflight_only=false`
|
||||
- `preflight_run_id=<successful preflight run id>`
|
||||
|
||||
CLI release notes:
|
||||
|
||||
- Real publishes are manual-only and require the workflow to be started from `main`.
|
||||
- The publish job waits at the GitHub `npm-release` environment for approval.
|
||||
- npm auth is handled through npm trusted publishing, not an `NPM_TOKEN`.
|
||||
- npm trusted publisher must be configured for package `clawhub` with repository `openclaw/clawhub`, workflow `clawhub-cli-npm-release.yml`, and environment `npm-release`.
|
||||
|
||||
That workflow assumes Vercel Git integration is enabled for this repo. It does
|
||||
not run `vercel deploy` directly; instead it waits for the GitHub commit status
|
||||
`Vercel – clawhub` for the pushed SHA, then runs smoke tests against
|
||||
production.
|
||||
not run `vercel deploy` directly; frontend-related steps wait for the GitHub
|
||||
commit status `Vercel – clawhub` for the selected SHA, then run smoke tests
|
||||
against production.
|
||||
|
||||
Ensure Convex env is set (auth + embeddings):
|
||||
|
||||
|
||||
@@ -8,6 +8,24 @@ read_when:
|
||||
|
||||
# GitHub import (public repos)
|
||||
|
||||
## CLI
|
||||
|
||||
For plugin authors, the recommended GitHub import path is now the CLI:
|
||||
|
||||
```bash
|
||||
clawhub package publish owner/repo
|
||||
clawhub package publish owner/repo@v1.0.0
|
||||
clawhub package publish https://github.com/owner/repo
|
||||
|
||||
# Preview only
|
||||
clawhub package publish owner/repo --dry-run
|
||||
|
||||
# CI-friendly output
|
||||
clawhub package publish owner/repo --dry-run --json
|
||||
```
|
||||
|
||||
This keeps package metadata zero-config where possible and auto-populates GitHub provenance.
|
||||
|
||||
Goal: paste a GitHub URL → auto-detect skill → preview files → publish (selective) → persist provenance.
|
||||
|
||||
Non-goal (v1): private repos (no OAuth/PAT support).
|
||||
|
||||
+18
-12
@@ -21,9 +21,9 @@ Enforcement model:
|
||||
- Authenticated requests (valid Bearer token): enforced per user bucket.
|
||||
- If token is missing/invalid, behavior falls back to IP enforcement.
|
||||
|
||||
- Read: 120/min per IP, 600/min per key
|
||||
- Write: 30/min per IP, 120/min per key
|
||||
- Download: 20/min per IP, 120/min per key (`/api/v1/download`)
|
||||
- Read: 180/min per IP, 900/min per key
|
||||
- Write: 45/min per IP, 180/min per key
|
||||
- Download: 30/min per IP, 180/min per key (`/api/v1/download`)
|
||||
|
||||
Headers:
|
||||
|
||||
@@ -237,6 +237,9 @@ Notes:
|
||||
|
||||
- If neither `version` nor `tag` is provided, uses the latest version.
|
||||
- Includes normalized verification status plus scanner-specific details.
|
||||
- `security.capabilityTags` includes deterministic capability/risk labels such as
|
||||
`crypto`, `requires-wallet`, `can-make-purchases`, `can-sign-transactions`,
|
||||
`requires-oauth-token`, and `posts-externally` when detected.
|
||||
- `security.hasScanResult` is `true` only when a scanner produced a definitive verdict (`clean`, `suspicious`, or `malicious`).
|
||||
- `moderation` is a current skill-level moderation snapshot derived from the latest version.
|
||||
- When querying a historical version, check `moderation.matchesRequestedVersion` and `moderation.sourceVersion` before treating `moderation` and `security` as the same version context.
|
||||
@@ -280,8 +283,8 @@ Notes:
|
||||
- Skill entries stay backed by the skill registry and can still be published only through `POST /api/v1/skills`.
|
||||
- `POST /api/v1/packages` is still only for code-plugin and bundle-plugin releases.
|
||||
- Anonymous callers only see public package channels.
|
||||
- Authenticated callers can see their own private packages in list/search results.
|
||||
- `channel=private` only returns packages owned by the authenticated caller.
|
||||
- Authenticated callers can see private packages for publishers they belong to in list/search results.
|
||||
- `channel=private` only returns packages the authenticated caller can read.
|
||||
|
||||
### `GET /api/v1/packages/search`
|
||||
|
||||
@@ -300,8 +303,8 @@ Query params:
|
||||
Notes:
|
||||
|
||||
- Anonymous callers only see public package channels.
|
||||
- Authenticated callers can search their own private packages.
|
||||
- `channel=private` only returns packages owned by the authenticated caller.
|
||||
- Authenticated callers can search private packages for publishers they belong to.
|
||||
- `channel=private` only returns packages the authenticated caller can read.
|
||||
|
||||
### `GET /api/v1/packages/{name}`
|
||||
|
||||
@@ -310,7 +313,7 @@ Returns package detail metadata.
|
||||
Notes:
|
||||
|
||||
- Skills can also resolve through this route in the unified catalog.
|
||||
- Private packages return `404` unless the caller is the owner.
|
||||
- Private packages return `404` unless the caller can read the owning publisher.
|
||||
|
||||
### `GET /api/v1/packages/{name}/versions`
|
||||
|
||||
@@ -323,15 +326,16 @@ Query params:
|
||||
|
||||
Notes:
|
||||
|
||||
- Private packages return `404` unless the caller is the owner.
|
||||
- Private packages return `404` unless the caller can read the owning publisher.
|
||||
|
||||
### `GET /api/v1/packages/{name}/versions/{version}`
|
||||
|
||||
Returns one package version, including file metadata, compatibility, capabilities, and verification.
|
||||
Returns one package version, including file metadata, compatibility, capabilities, verification, and scan data.
|
||||
|
||||
Notes:
|
||||
|
||||
- Private packages return `404` unless the caller is the owner.
|
||||
- `version.sha256hash`, `version.vtAnalysis`, `version.llmAnalysis`, and `version.staticScan` are included when scan data exists.
|
||||
- Private packages return `404` unless the caller can read the owning publisher.
|
||||
|
||||
### `GET /api/v1/packages/{name}/file`
|
||||
|
||||
@@ -349,7 +353,8 @@ Notes:
|
||||
- Uses the read rate bucket, not the download bucket.
|
||||
- Binary files return `415`.
|
||||
- File size limit: 200KB.
|
||||
- Private packages return `404` unless the caller is the owner.
|
||||
- Pending VirusTotal scans do not block reads; malicious releases may still be withheld elsewhere.
|
||||
- Private packages return `404` unless the caller can read the owning publisher.
|
||||
|
||||
### `GET /api/v1/packages/{name}/download`
|
||||
|
||||
@@ -366,6 +371,7 @@ Notes:
|
||||
- Skills redirect to `GET /api/v1/download`.
|
||||
- Plugin/package archives are zip files with a `package/` root so they install directly in OpenClaw without repacking.
|
||||
- Registry-only metadata is not injected into the downloaded archive.
|
||||
- Pending VirusTotal scans do not block downloads; malicious releases return `403`.
|
||||
- Private packages return `404` unless the caller is the owner.
|
||||
|
||||
### `GET /api/v1/resolve`
|
||||
|
||||
@@ -47,9 +47,9 @@ read_when:
|
||||
- `SKILL.md`
|
||||
- `notes.md`
|
||||
- Publish:
|
||||
- `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
|
||||
- `bun clawhub skill publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.0 --tags latest`
|
||||
- Publish update with empty changelog:
|
||||
- `bun clawhub publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
|
||||
- `bun clawhub skill publish . --slug clawhub-manual-<ts> --name "Manual <ts>" --version 1.0.1 --tags latest`
|
||||
|
||||
## Delete / undelete (owner/admin)
|
||||
|
||||
|
||||
@@ -485,4 +485,3 @@ Add or update tests for:
|
||||
drift
|
||||
- Do not keep slug-only and scoped lookup logic equally primary; one must win
|
||||
- Prefer publisher abstraction over `ownerUserId | ownerOrgId` unions
|
||||
|
||||
|
||||
+1
-1
@@ -98,7 +98,7 @@ EOF
|
||||
Publish:
|
||||
|
||||
```bash
|
||||
bun clawhub publish . \
|
||||
bun clawhub skill publish . \
|
||||
--slug clawhub-demo-$(date +%s) \
|
||||
--name "Demo $(date +%s)" \
|
||||
--version 1.0.0 \
|
||||
|
||||
@@ -8,6 +8,8 @@ read_when:
|
||||
|
||||
# Security + Moderation
|
||||
|
||||
See also: [acceptable-usage.md](./acceptable-usage.md) for the marketplace policy on prohibited skill categories.
|
||||
|
||||
## Roles + permissions
|
||||
|
||||
- user: upload skills/souls (subject to GitHub age gate), report skills/comments.
|
||||
@@ -42,6 +44,11 @@ read_when:
|
||||
## Skill moderation pipeline
|
||||
|
||||
- New skill publishes now persist a deterministic static scan result on the version.
|
||||
- Package/plugin scan backfills now also recompute deterministic static scan results for older releases,
|
||||
so legacy plugin versions can surface OpenClaw scan findings without republishing.
|
||||
- Source-linked packages can fall back to a clean package verdict when VirusTotal only returns
|
||||
undetected engine results, provided the LLM scan is clean and static scan is non-malicious. This
|
||||
avoids indefinite pending scans when VT Code Insight never materializes.
|
||||
- Skill moderation state stores a structured snapshot:
|
||||
- `moderationVerdict`: `clean | suspicious | malicious`
|
||||
- `moderationReasonCodes[]`: canonical machine-readable reasons
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
# Logo Replacement Design
|
||||
|
||||
Date: 2026-04-21
|
||||
Topic: Comprehensive logo replacement using the provided lobster artwork
|
||||
|
||||
## Summary
|
||||
|
||||
Replace every current application logo surface with the user-provided lobster artwork while preserving the existing UI layout and copy. This includes in-app logo images, favicon and install icon assets, and manifest/head wiring. The existing wide social preview image `public/og.png` remains unchanged. Instead, `public/og-logo.png` is included in the replacement asset pack as a standalone logo export and is not wired into site metadata.
|
||||
|
||||
## Goals
|
||||
|
||||
- Replace all current logo imagery with the provided lobster art.
|
||||
- Preserve existing layout structure in header, mobile navigation, and hero content.
|
||||
- Provide dedicated asset files for browser, install, and app surfaces rather than relying on one large source image everywhere.
|
||||
- Keep runtime references stable where possible by replacing existing filenames in place.
|
||||
- Improve browser/device logo behavior by adding standard favicon and touch icon variants.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- No header, navigation, or hero layout redesign.
|
||||
- No typography or copy changes to the `ClawHub` wordmark text.
|
||||
- No change to the existing social preview card asset `public/og.png`.
|
||||
- No full vector redraw of the lobster artwork from scratch.
|
||||
|
||||
## Scope
|
||||
|
||||
### In Scope
|
||||
|
||||
- Replace:
|
||||
- `public/clawd-logo.png`
|
||||
- `public/clawd-mark.png`
|
||||
- `public/logo192.png`
|
||||
- `public/logo512.png`
|
||||
- `public/favicon.ico`
|
||||
- Add or update:
|
||||
- `public/favicon-16x16.png`
|
||||
- `public/favicon-32x32.png`
|
||||
- `public/apple-touch-icon.png`
|
||||
- `public/logo.jpg`
|
||||
- `public/logo.svg`
|
||||
- `public/og-logo.png`
|
||||
- Update runtime/browser metadata:
|
||||
- root document link tags in `src/routes/__root.tsx`
|
||||
- `public/manifest.json`
|
||||
|
||||
### Out of Scope
|
||||
|
||||
- `public/og.png`
|
||||
- Any route-level social metadata currently using `og.png`
|
||||
- Any non-logo artwork or unrelated illustration assets
|
||||
|
||||
## Current State
|
||||
|
||||
- The app currently references `public/clawd-logo.png` in the desktop and mobile header.
|
||||
- The homepage hero references `public/clawd-mark.png`.
|
||||
- The root document exposes `/favicon.ico`, `/logo192.png`, and `/manifest.json`.
|
||||
- The web app manifest references `favicon.ico`, `logo192.png`, and `logo512.png`.
|
||||
- The site-wide OG metadata still references `og.png`.
|
||||
|
||||
## Recommended Approach
|
||||
|
||||
Use the provided lobster image as the master artwork and derive a small asset pack tailored to each output surface.
|
||||
|
||||
Why this approach:
|
||||
|
||||
- It satisfies the request to replace the logo everywhere it appears.
|
||||
- It avoids visual degradation from blindly reusing one oversized raster in tiny favicon contexts.
|
||||
- It minimizes application code changes by preserving the established filenames used by the UI.
|
||||
|
||||
## Asset Plan
|
||||
|
||||
### Master Asset
|
||||
|
||||
Create one high-resolution square source derived from the attached lobster artwork. This will be the basis for all exported formats.
|
||||
|
||||
### Replacement Assets
|
||||
|
||||
- `clawd-logo.png`
|
||||
- High-resolution square PNG used by header/mobile brand image references.
|
||||
- `clawd-mark.png`
|
||||
- High-resolution square PNG used by hero/logo-only surfaces.
|
||||
- `logo192.png`
|
||||
- 192×192 install icon.
|
||||
- `logo512.png`
|
||||
- 512×512 install icon.
|
||||
- `favicon.ico`
|
||||
- Multi-size favicon generated from the same master for browser tab use.
|
||||
- `favicon-16x16.png`
|
||||
- Explicit raster favicon for browsers that prefer PNG.
|
||||
- `favicon-32x32.png`
|
||||
- Explicit raster favicon for higher-density tab/bookmark use.
|
||||
- `apple-touch-icon.png`
|
||||
- 180×180 touch icon for iOS home screen usage.
|
||||
- `logo.jpg`
|
||||
- Flattened JPEG export for contexts where a non-transparent logo file is useful.
|
||||
- `logo.svg`
|
||||
- SVG wrapper asset that embeds the logo image in an SVG container so an SVG logo file exists for downstream usage without falsely claiming the art is natively vector.
|
||||
- `og-logo.png`
|
||||
- Logo-focused branded raster asset retained separately from the existing wide social card `og.png`.
|
||||
|
||||
## Runtime Wiring
|
||||
|
||||
### Application UI
|
||||
|
||||
- Keep existing JSX references to `clawd-logo.png` and `clawd-mark.png` unless a clearer dedicated asset path becomes necessary.
|
||||
- Do not replace image elements with text or SVG components.
|
||||
|
||||
### Root Head Tags
|
||||
|
||||
Update `src/routes/__root.tsx` to use dedicated icon assets:
|
||||
|
||||
- `rel="icon"` should include PNG favicon variants in addition to the ICO.
|
||||
- `rel="apple-touch-icon"` should point to `apple-touch-icon.png`.
|
||||
- `rel="manifest"` remains `manifest.json`.
|
||||
- OG/Twitter metadata remains wired to `og.png` and is not changed.
|
||||
|
||||
### Web App Manifest
|
||||
|
||||
Update `public/manifest.json` so install surfaces reference the replacement icon assets. Keep the manifest conservative and omit maskable-specific `purpose` values for this change.
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. Start from the provided lobster artwork.
|
||||
2. Export optimized raster variants for each target size.
|
||||
3. Replace or add files in `public/`.
|
||||
4. Update root document links and manifest entries.
|
||||
5. Build the app and verify the logo surfaces still render without layout regressions.
|
||||
|
||||
## Error Handling And Risks
|
||||
|
||||
### Small-Size Legibility
|
||||
|
||||
Risk: the artwork is detailed and may lose clarity at favicon sizes.
|
||||
|
||||
Mitigation:
|
||||
|
||||
- Generate dedicated 16×16 and 32×32 outputs instead of relying only on browser downscaling.
|
||||
- Prefer the ICO plus PNG favicon set to maximize compatibility.
|
||||
|
||||
### Raster-As-Vector Expectations
|
||||
|
||||
Risk: a pure SVG redraw would be time-consuming and subjective.
|
||||
|
||||
Mitigation:
|
||||
|
||||
- Provide `logo.svg` as an SVG container asset, while using raster files for browser/runtime surfaces that need visual fidelity.
|
||||
|
||||
### Unintended Social Preview Changes
|
||||
|
||||
Risk: a broad asset refresh accidentally changes OG behavior.
|
||||
|
||||
Mitigation:
|
||||
|
||||
- Explicitly leave `og.png` and its metadata references untouched.
|
||||
- Treat `og-logo.png` as a separate logo asset only.
|
||||
|
||||
## Testing And Verification
|
||||
|
||||
- Confirm the generated files exist in `public/` with expected dimensions.
|
||||
- Run the production build to ensure asset references still resolve.
|
||||
- Spot-check the following surfaces:
|
||||
- desktop header brand image
|
||||
- mobile navigation brand image
|
||||
- homepage hero lobster image
|
||||
- browser favicon and touch icon wiring
|
||||
- manifest icon references
|
||||
- Verify that `og.png` remains unchanged and the site metadata still references it.
|
||||
|
||||
## Implementation Notes
|
||||
|
||||
- Use minimal code churn: replace files in place where existing paths are already correct.
|
||||
- Add new icon files only where they improve browser/device handling.
|
||||
- Keep the change tightly scoped to branding assets and metadata.
|
||||
|
||||
## Acceptance Criteria
|
||||
|
||||
- Every current application logo surface displays the provided lobster artwork instead of the previous brand image.
|
||||
- Favicon, touch icon, and install icons resolve to replacement assets.
|
||||
- Header/mobile/hero layout remains unchanged.
|
||||
- `og.png` is not modified.
|
||||
- `og-logo.png` exists as part of the updated asset pack.
|
||||
- The app builds successfully after the change.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user