clawhub/specs/github-backed-skills.md

GitHub-backed skills

GitHub-backed skills are source-backed ClawHub catalog entries. ClawHub indexes metadata, security state, and display content, but the install artifact remains the upstream GitHub repository at a specific commit.

This exists for trusted upstream catalogs such as NVIDIA/skills, where the publisher wants one canonical source of truth and does not want ClawHub to republish byte copies as if they were ClawHub-owned artifacts.

Model

There are two ClawHub skill models:

• Hosted upload: ClawHub stores versioned skill artifacts and installs from ClawHub bytes.
• GitHub-backed: ClawHub stores catalog metadata and cached Markdown for display, then install/update resolves to GitHub source bytes at a commit.

For GitHub-backed skills, skills.latestVersionId is normally absent. The real install identity is the GitHub commit and skill-folder content hash:

• githubCurrentCommit: upstream repo commit currently known by ClawHub
• githubCurrentContentHash: hash of the skill folder content at that commit
• githubScanStatus: ClawHub security result for that exact content hash

If an upstream manifest includes a version-like field for a skill, ClawHub may display it, but it is not the install identity. The commit is the install version.

Source Tables

githubSkillSources represents a synced GitHub skills repository:

• repo: GitHub owner/repo, for example NVIDIA/skills
• ownerPublisherId: ClawHub publisher that owns the synced catalog
• defaultBranch: optional branch value discovered from GitHub
• displayManifestKind: currently only skills.sh
• displayManifestCommit, displayManifestHash, displayManifestFetchedAt
• displayManifestStatus and displayManifest
• sync status and invalid-skill diagnostics

githubSkillContents stores display-only content for individual source-backed skills:

• skillMarkdown and source path for SKILL.md
• optional skillCardMarkdown and source path for skill-card.md
• githubCommit and githubContentHash that the cached content came from

This table is not an install artifact store. OpenClaw must not install from githubSkillContents.

githubSkillCandidates stores an exact pending replacement for a canonical skill while its currently allowed source remains active:

• canonical skillId
• immutable source repository, path, commit, and folder content hash
• bounded display Markdown fetched from that exact commit
• the ClawHub scan state for that exact content

Candidates do not create skillVersions. A clean or suspicious exact verdict promotes the candidate onto the existing skill row and then deletes the candidate. Failed, malicious, stale, removed, or disconnected candidates never replace the active source.

Dark skills.sh discovery metadata

The skills.sh discovery pipeline has a separate hidden metadata table and does not create or mutate skills rows during its planning gates.

• A discovery row keeps the immutable GitHub owner ID plus exact repository, path, commit, folder content hash, source URL, and source snapshot.
• Native reconciliation is advisory metadata only. It classifies an observation as new, an exact native GitHub-source match, or a slug/route collision.
• Exact matches may record the native download count for readback, but discovery must not patch that count or any other native field.
• A verified GitHub owner may create a claim opportunity marker. That marker is not publisher attachment, ownership, profile content, or claim execution.
• Hidden discovery rows are never installable and cannot become public through the discovery control plane.
• The permanent Test canary uses one committed skills.sh observation fixture, while GitHub owner, repository, commit, path, and content verification remain live and authenticated.
• Canary rollback may delete only the controlled hidden fixture row. It must leave native skills, download history, scan jobs, publishers, and aliases unchanged.

skills stores the public catalog row and install state:

• installKind: "github"
• githubSourceId
• githubPath
• githubHasSkillCard
• githubCurrentCommit
• githubCurrentContentHash
• githubCurrentStatus: present, missing, or unknown
• githubCurrentCheckedAt
• githubScanStatus: pending, clean, suspicious, malicious, or failed
• githubRemovedAt

Permanent external skills.sh mirror

The full authenticated skills.sh mirror is separate from both native skills and the controlled scan-admission catalog. It stores source observations in skillsShMirrorDigests and bounded detail content in skillsShMirrorDetails; controls, durable run cursors, and conflicts remain in their own mirror tables.

• GitHub identities are exact owner/repo/slug values. Well-known identities are exact sourceHost/slug values and must not invent a repository owner.
• Every digest is permanently publicVisible: false and installable: false. Mirror ingestion never creates native skills, publisher attachment, claims, scan plans, or scan jobs.
• The digest stores normalized slug/display-name fields and a lean searchText. Exact, prefix, first-token, popularity, freshness, and full-text indexes are staged before activation on the permanent Test corpus.
• Gen Agent Trust Hub, Socket, and Snyk observations are stored independently with a bounded status plus optional source timestamp and source link. These are upstream claims only and must never be serialized as a ClawHub verdict.
• Detail storage retains at most one preferred SKILL.md or README.md, capped at 64 KiB. The complete upstream file tree is never persisted for an unclaimed mirror row.
• Snapshot ingestion uses bounded page/offset cursors. Pause is checked before another source batch is fetched; resume continues at the exact stored cursor. Reconciliation tombstones disappeared rows and reactivates later observations without deleting or changing native skills.
• The mirror has no scheduler in this stage. Production activation, public search/detail/install behavior, claims, and publisher attachment require separately accepted work.

Publisher Gate

The legacy NVIDIA/skills source keeps its existing production behavior. Generic GitHub Skill Sync remains dark unless the GitHub Skill Sync rollout capability is enabled.

When enabled, repository enrollment requires:

• publisher-admin access
• a public GitHub repository
• immutable GitHub repository and owner IDs
• the publisher's matching immutable GitHub user ID, or a verified GitHub organization ID plus fresh organization-admin membership

Repository names, redirects, display handles, and skills.sh owner strings do not authorize enrollment. The repository identity is checked again after snapshot fetch so a transfer cannot race source activation.

Sync

ClawHub owns the sync loop. Upstream repositories do not push payloads into ClawHub.

The production cron runs every 15 minutes:

github-skill-source-sync -> githubSkillSyncNode.syncGitHubSkillSourcesInternal

That cron runs in Convex's Node runtime because fetching and expanding a source archive can exceed the default action runtime's memory limit. It fetches the current public GitHub repo, reads skills.sh.json, builds a source snapshot, and applies it to ClawHub. Pagination must use a stable source cursor, not updatedAt, because syncing a row updates the row. Cursor continuations must remain on the Node runtime action. Per-skill verification also fetches and expands the source archive, so verification actions must run in the Node runtime as well.

Sync must not pass the full repo Markdown payload through one large Convex mutation. The intended split is:

1. Apply source metadata and skill state.
2. Fetch small target rows for changed/current skills.
3. Persist SKILL.md / skill-card.md content per skill.

Source discovery treats the repo's skills/ tree as the canonical catalog area. If package directories also contain copied skill folders that normalize to the same slug, and exactly one matching SKILL.md path lives under skills/, sync uses that catalog path. If two or more catalog paths normalize to the same slug, sync rejects the repo with a client-visible validation error.

Manifest Rendering

The only supported display manifest today is skills.sh.json.

ClawHub parses it into the structured displayManifest shape used by publisher profile rendering. The UI should not interpret arbitrary raw manifest JSON at render time.

Unsupported, invalid, or missing manifest data should not block the source sync. The catalog can still render skills in the normal fallback order.

Repo labels such as NVIDIA/skills and "Source-backed" chrome are implementation details and should not be required visible UI copy.

Security Invariants

GitHub-backed security scanning follows the same core invariant as hosted uploads:

A normal install/update may only install content whose exact current content hash has a completed, non-blocked ClawHub scan result.

When a new source-backed skill appears:

• set githubScanStatus: "pending"
• keep the catalog entry visible with moderationReason: "pending.scan", while blocking normal install/update until the full scan completes
• fetch the exact skill-folder bytes for the current commit and content hash
• store those bytes only in ephemeral Convex storage, referenced by the skillScanRequests row through a prepare, bounded chunk append, and finalize sequence so no action-to-mutation argument carries the full file manifest; create the request before storing blobs and persist ownership after each bounded chunk so a terminated action can orphan at most its current chunk
• cap the persisted file manifest at 4 MiB of descriptor metadata and hydrate one signed-URL-heavy worker job per claim response
• enqueue the normal full ClawScan worker with deterministic static findings as input context
• do not schedule another heavy verification action while that content hash already has an active queued/running scan job or a recently prepared request
• enqueue explicit owner/moderator rescans in the high-priority manual queue

When an already allowed GitHub-backed skill changes, or a Hosted Skill owned by the same publisher occupies the destination, ClawHub creates an exact pending candidate instead of overwriting the active source. The prior verified GitHub commit or hosted version remains active until the candidate's own exact scan is allowed. Promotion patches the existing canonical skill row in place, preserving its slug, routes, ClawHub metrics, bookmarks, prior hosted versions, and audit relationships.

When verification succeeds cleanly:

• persist the completed ClawScan, SkillSpector, and static findings on a durable githubSkillScans row keyed by skill and content hash
• set githubScanStatus: "clean"
• make the skill active/installable

When verification is suspicious:

• persist the final result on the same durable content-hash scan row
• set githubScanStatus: "suspicious"
• keep the skill active/installable with suspicious review metadata

When verification fails or is malicious:

• persist the final result on the same durable content-hash scan row
• keep/block the skill from normal install
• return a structured install block such as github_scan_failed

Completed clean, suspicious, and malicious verdicts may be reused for the same skill and content hash. Failed worker runs are not reusable verdicts and may be requeued for that same content hash after the underlying runtime problem is fixed. Reusing a result must reassociate it with the skill's current source, commit, and path before any old-source cleanup can run.

Legacy GitHub-backed rows that have a scan status but no durable githubSkillScans result are not trusted as full ClawScan verdicts. The next source sync must move them back to pending and enqueue the full pipeline.

GitHub-backed verification must not create a hosted skillVersions row or a ClawHub-owned install artifact. Expired request rows and their temporary stored files/chunks are pruned, while the small completed githubSkillScans result remains available for the public Security audit page. Source-wide scan-history cleanup runs in bounded asynchronous batches. Expired request cleanup deletes the linked worker job before deleting any files, then deletes at most one bounded GitHub file-metadata chunk per request and schedules an immediate continuation while work remains. Static findings alone must never promote or block the skill; the full ClawScan verdict controls githubScanStatus.

If the upstream path disappears:

• set githubCurrentStatus: "missing"
• set githubRemovedAt
• hide the skill with moderationReason: "github.upstream.removed"
• block install/update with github_upstream_removed or github_upstream_missing

The row may remain for audit/history, but users must not silently install an old ClawHub-cached revision after upstream removed or changed it.

Removing a selected repository also removes its pending candidates before deleting the source, so an in-flight callback cannot promote disconnected content. Active source-backed skills become missing and hidden. Re-enrollment or upstream reappearance may revive the same canonical skill row, but the reappeared exact content returns to pending and must pass scanning before installation.

Install Resolver

Normal install/update resolves through ClawHub:

OpenClaw -> ClawHub install resolver -> pinned GitHub descriptor

For GitHub-backed skills with a completed non-blocked scan result, ClawHub returns:

{
  "ok": true,
  "installKind": "github",
  "github": {
    "repo": "NVIDIA/skills",
    "path": "skills/aiq-deploy",
    "commit": "<40-char sha>",
    "contentHash": "<skill-folder hash>",
    "sourceUrl": "https://github.com/NVIDIA/skills/tree/<sha>/skills/aiq-deploy"
  }
}

OpenClaw downloads the GitHub archive for that commit and extracts only the skill path. The local lock/origin version is the commit SHA.

Controlled unclaimed skills.sh catalog entries use the repository-qualified reference skills-sh/<owner>/<repo>/<slug>. The colon form skills-sh:<owner>/<repo>/<slug> is invalid and must be rejected by clients and HTTP handlers.

In Local and the permanent Test environment, an unclaimed catalog entry remains hidden and non-installable until a real low-priority ClawHub scan completes for the exact catalog identity, immutable GitHub owner ID, repository, path, commit, folder content hash, source content hash, and scan attempt. Clean and suspicious verdicts may publish only that exact attempt. Malicious, failed, canceled, or stale callbacks cannot publish it.

The public Test route is /skills-sh/<owner>/<repo>/<slug>. Its install resolver returns the same commit-pinned GitHub descriptor used by native GitHub-backed skills. Catalog pause, kill, publication disable, and exact-attempt rollback must fail closed without disabling or mutating native scan work.

Pending verification keeps the skill visible in ClawHub search and detail UI, but normal install/update returns a structured block:

{
  "ok": false,
  "reason": "github_verification_pending",
  "status": 423,
  "message": "GitHub-backed skill security scan is in progress. Try again shortly, or rerun with --force-install to install the unverified upstream commit."
}

--force-install may bypass only pending GitHub-backed verification. It must not bypass failed, malicious, missing, or removed upstream states.

No Mirror Contract

ClawHub must not create hosted skillVersions or ClawHub download artifacts for GitHub-backed skills.

GET /api/v1/download may still be used as a metered source handoff for current GitHub-backed skills whose scan verdict is clean or suspicious. In that case ClawHub returns only stored fetch coordinates (sourceRef: "public-github", repo, commit, path, content hash, and an archive URL) after checking current upstream and scan state. It must not fetch GitHub, expand archives, proxy bytes, create skillVersions, or include detailed scan metadata in the successful payload.

GET /api/v1/skills/export follows the same no-mirror contract. Hosted skills continue to export stored version files with sourceRef: "public-clawhub". Current GitHub-backed skills whose scan verdict is clean or suspicious are included as sourceRef: "public-github" entries with _source_handoff.json coordinates, not ClawHub-hosted source files.

This avoids two NVIDIA concerns:

• Signature drift: any byte-level transformation in a mirror can invalidate upstream detached signatures.
• Stale security chain: if a mirror lags after upstream update/removal, users might silently install old bytes.

Because installs use GitHub source at a specific commit, ClawHub does not claim to preserve or verify upstream OMS signatures in v1. Signature verification can be added later as an additional verification input, but it must not require ClawHub to republish transformed skill bytes.

UI

Publisher profiles may group GitHub-backed skills by the parsed skills.sh.json manifest. Skill detail pages read cached SKILL.md and optional skill-card.md from githubSkillContents.

UI display state is advisory. Installability is controlled by the install resolver and current scan/upstream state, not by stale cached UI metadata.

Testing Expectations

Keep coverage for:

• immutable publisher/repository authorization and transfer-race rejection
• parsing skills.sh.json
• 15-minute cron registration
• new skill -> pending scan -> blocked install
• changed content hash -> pending candidate -> prior verified commit remains served
• Hosted Skill -> pending candidate -> in-place GitHub promotion with history preserved
• reusable exact verdict -> promotion only after exact display content is cached
• clean verification -> pinned GitHub install descriptor
• suspicious verification -> pinned GitHub install descriptor with suspicious review metadata
• failed/malicious scan -> blocked install
• removed upstream path -> hidden/blocked install -> scan-gated reappearance
• repository removal -> pending candidates canceled and active source blocked
• cached SKILL.md and skill-card.md display content
• no skillVersions for GitHub-backed skills
• OpenClaw installing the pinned GitHub commit/path rather than ClawHub bytes