docs: add Mintlify-ready docs set

This commit is contained in:
Peter Steinberger
2026-01-07 17:52:55 +01:00
parent 974338fb97
commit 1bf3bcf152
10 changed files with 700 additions and 0 deletions
+31
View File
@@ -0,0 +1,31 @@
---
title: 'Docs'
description: 'ClawdHub documentation index + reading order.'
---
# Docs
Reading order (new contributor):
1. `README.md` (repo root): run locally.
2. `docs/quickstart.md`: end-to-end: search → install → publish → sync.
3. `docs/architecture.md`: how the pieces fit (TanStack Start + Convex + CLI).
4. `docs/skill-format.md`: what a “skill” is on disk + on the registry.
5. `docs/cli.md`: CLI reference (flags, config, lockfiles, sync rules).
6. `docs/http-api.md`: HTTP endpoints used by the CLI + public API.
7. `docs/auth.md`: GitHub OAuth + API tokens + CLI loopback login.
8. `docs/deploy.md`: Convex + Vercel deployment + rewrites.
9. `docs/troubleshooting.md`: common failure modes.
Feature/ops docs (already present):
- `docs/spec.md`: product + implementation spec (data model + flows).
- `docs/telemetry.md`: what `clawdhub sync` reports; opt-out.
- `docs/webhook.md`: Discord webhook events/payload.
- `docs/diffing.md`: version-to-version diff UI spec.
- `docs/manual-testing.md`: CLI smoke scripts.
Docs tooling:
- `docs/mintlify.md`: publish these docs with Mintlify.
+61
View File
@@ -0,0 +1,61 @@
---
title: 'Architecture'
description: 'How the web app, Convex backend, and CLI interact.'
---
# Architecture
## Pieces
- Web app: TanStack Start (React) under `src/`.
- Backend: Convex under `convex/` (DB, storage, actions, HTTP routes).
- CLI: `packages/clawdhub/` (published as `clawdhub`).
- Shared schemas/routes: `packages/schema/` (`clawdhub-schema`).
## Data + storage
- Skill “bundle” = versioned set of text files stored in Convex `_storage`.
- Metadata extracted from `SKILL.md` frontmatter.
- Stats stored on `skills` (downloads, installs, stars, comments, …).
## Main flows
### Browse (web)
- UI reads skill metadata + latest version from Convex queries/actions.
- `SKILL.md` rendered as Markdown.
### Search (HTTP)
- `/api/search?q=...` routes to Convex action for vector search.
- Embeddings currently generated during publish.
### Install (CLI)
- Resolve latest version via `/api/skill?slug=...`.
- Download zip via `/api/download?slug=...&version=...`.
- Extract into `./skills/<slug>` (default).
- Persist install state:
- `./.clawdhub/lock.json` (per workdir)
- `./skills/<slug>/.clawdhub/origin.json` (per skill folder)
### Update (CLI)
- Hash local files, call `/api/skill/resolve?slug=...&hash=<sha256>`.
- If local matches a known version → use that for “current”.
- If local doesnt match:
- refuse by default
- or overwrite with `--force`
### Publish (CLI)
- Upload each text file via `/api/cli/upload-url` (Convex upload URL).
- Publish metadata via `/api/cli/publish` (requires Bearer token).
### Sync (CLI)
- Scan roots for skill folders (contain `SKILL.md`).
- Compute fingerprint; compare to registry state.
- Optionally reports telemetry (see `docs/telemetry.md`).
- Publishes new/changed skills (skips modified installed skills inside install root).
+53
View File
@@ -0,0 +1,53 @@
---
title: 'Auth'
description: 'GitHub OAuth (web) + API tokens (CLI).'
---
# Auth
## Web auth (GitHub OAuth)
- Convex Auth + GitHub OAuth App.
- Env vars:
- `AUTH_GITHUB_ID`
- `AUTH_GITHUB_SECRET`
- `CONVEX_SITE_URL` (used by auth config)
Local setup steps are in the repo root `README.md`.
## API tokens (CLI)
The CLI uses a long-lived API token (Bearer token) for publish/sync/delete.
### Browser flow (default)
`clawdhub login` does:
1. Starts a loopback HTTP server on `127.0.0.1` (random port).
2. Opens `<site>/cli/auth?redirect_uri=http://127.0.0.1:<port>/callback&state=...`.
3. Web UI requires GitHub login, then creates a token and redirects back to the loopback server.
4. CLI stores the token in the global config file.
### Headless flow
Create a token in the web UI (Settings → API tokens) and paste it:
```bash
clawdhub login --token clh_...
```
### Token storage
Default global config path:
- macOS: `~/Library/Application Support/clawdhub/config.json`
Override:
- `CLAWDHUB_CONFIG_PATH=/path/to/config.json`
### Revocation
- Tokens can be revoked in the web UI.
- Revoked tokens return `401 Unauthorized` on CLI endpoints.
+94
View File
@@ -0,0 +1,94 @@
---
title: 'CLI'
description: 'Commands, flags, config files, and state on disk.'
---
# CLI
CLI package: `packages/clawdhub/` (bin: `clawdhub`).
From this repo you can run it via the wrapper script:
```bash
bun clawdhub --help
```
## Global flags
- `--workdir <dir>`: working directory (default: cwd)
- `--dir <dir>`: install dir under workdir (default: `skills`)
- `--site <url>`: base URL for browser login (default: `https://clawdhub.com`)
- `--registry <url>`: API base URL (default: discovered, else `https://clawdhub.com`)
- `--no-input`: disable prompts
Env equivalents:
- `CLAWDHUB_SITE`
- `CLAWDHUB_REGISTRY`
## Config file
Stores your API token + cached registry URL.
- macOS: `~/Library/Application Support/clawdhub/config.json`
- override: `CLAWDHUB_CONFIG_PATH`
## Commands
### `login` / `auth login`
- Default: opens browser to `<site>/cli/auth` and completes via loopback callback.
- Headless: `clawdhub login --token clh_...`
### `whoami`
- Verifies the stored token via `/api/cli/whoami`.
### `search <query...>`
- Calls `/api/search?q=...`.
### `install <slug>`
- Resolves latest version via `/api/skill?slug=...`.
- Downloads zip via `/api/download`.
- Extracts into `<workdir>/<dir>/<slug>`.
- Writes:
- `<workdir>/.clawdhub/lock.json`
- `<skill>/.clawdhub/origin.json`
### `list`
- Reads `<workdir>/.clawdhub/lock.json`.
### `update [slug]` / `update --all`
- Computes fingerprint from local files.
- If fingerprint matches a known version: no prompt.
- If fingerprint does not match:
- refuses by default
- overwrites with `--force` (or prompt, if interactive)
### `publish <path>`
- Uploads each file via `/api/cli/upload-url`.
- Publishes via `/api/cli/publish`.
- Requires semver: `--version 1.2.3`.
### `sync`
- Scans for local skill folders and publishes new/changed ones.
- Flags:
- `--root <dir...>` extra scan roots
- `--all` upload without prompting
- `--dry-run` show plan only
- `--bump patch|minor|major` (default: patch)
- `--changelog <text>` (non-interactive)
- `--tags a,b,c` (default: latest)
- `--concurrency <n>` (default: 4)
Telemetry:
- Sent during `sync` when logged in, unless `CLAWDHUB_DISABLE_TELEMETRY=1`.
- Details: `docs/telemetry.md`.
+77
View File
@@ -0,0 +1,77 @@
---
title: 'Deploy'
description: 'Deploy web app + Convex backend, wire /api rewrites, set env.'
---
# Deploy
ClawdHub is two deployables:
- Web app (TanStack Start) → typically Vercel.
- Convex backend → Convex deployment (serves `/api/...` routes).
## 1) Deploy Convex
From your local machine:
```bash
bunx convex deploy
```
Ensure Convex env is set (auth + embeddings):
- `AUTH_GITHUB_ID`
- `AUTH_GITHUB_SECRET`
- `CONVEX_SITE_URL`
- `JWT_PRIVATE_KEY`
- `JWKS`
- `OPENAI_API_KEY`
- `SITE_URL` (your web app URL)
- Optional webhook env (see `docs/webhook.md`)
## 2) Deploy web app (Vercel)
Set env vars:
- `VITE_CONVEX_URL`
- `VITE_CONVEX_SITE_URL` (Convex “site” URL)
- `CONVEX_SITE_URL` (same value; used by auth provider config)
- `SITE_URL` (web app URL)
## 3) Route `/api/*` to Convex
This repo currently uses `vercel.json` rewrites:
- `source: /api/:path*`
- `destination: https://<deployment>.convex.site/api/:path*`
For self-host:
- update `vercel.json` to your deployments Convex site URL.
## 4) Registry discovery
The CLI can discover the API base from:
- `/.well-known/clawdhub.json`
If you dont serve that file, users must set:
```bash
export CLAWDHUB_REGISTRY=https://your-site.example
```
## 5) Post-deploy checks
```bash
curl -i "https://<site>/api/search?q=test"
curl -i "https://<site>/api/skill?slug=gifgrep"
```
Then:
```bash
clawdhub login --site https://<site>
clawdhub whoami
```
+122
View File
@@ -0,0 +1,122 @@
---
title: 'HTTP API'
description: 'Public endpoints + CLI endpoints (token auth).'
---
# HTTP API
Base URL: `https://clawdhub.com` (default).
All paths below are under `/api/...` and implemented by Convex HTTP routes (`convex/http.ts`).
## Public endpoints (no auth)
### `GET /api/search`
Query params:
- `q` (required): query string
- `limit` (optional): integer
- `approvedOnly` (optional): `true` to filter to approved-only skills (server may treat as “approved”/badged)
Response:
```json
{ "results": [{ "score": 0.123, "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "version": "1.2.3", "updatedAt": 1730000000000 }] }
```
### `GET /api/skill`
Query params:
- `slug` (required)
Response (shape is stable; contents may expand):
```json
{ "skill": { "slug": "gifgrep", "displayName": "GifGrep", "summary": "…", "tags": { "latest": "…" }, "stats": {}, "createdAt": 0, "updatedAt": 0 }, "latestVersion": { "version": "1.2.3", "createdAt": 0, "changelog": "…" }, "owner": { "handle": "steipete", "displayName": "Peter", "image": null } }
```
### `GET /api/skill/resolve`
Used by the CLI to map a local fingerprint to a known version.
Query params:
- `slug` (required)
- `hash` (required): 64-char hex sha256 of the bundle fingerprint
Response:
```json
{ "slug": "gifgrep", "match": { "version": "1.2.2" }, "latestVersion": { "version": "1.2.3" } }
```
### `GET /api/download`
Downloads a zip of a skill version.
Query params:
- `slug` (required)
- `version` (optional): semver string
- `tag` (optional): tag name (e.g. `latest`)
Notes:
- If neither `version` nor `tag` is provided, the latest version is used.
- Soft-deleted versions return `410`.
## CLI endpoints (Bearer token)
All CLI endpoints require:
```
Authorization: Bearer clh_...
```
### `GET /api/cli/whoami`
Validates token and returns the user handle.
### `POST /api/cli/upload-url`
Returns a Convex upload URL for a single file upload.
Response:
```json
{ "uploadUrl": "https://..." }
```
### `POST /api/cli/publish`
Publishes a new version from uploaded files.
- Validates semver, slug, size limits, text-only files, and `SKILL.md`.
- Generates embeddings (requires `OPENAI_API_KEY` server-side).
### `POST /api/cli/telemetry/sync`
Used by `clawdhub sync` to report install telemetry.
Details: `docs/telemetry.md`.
### `POST /api/cli/skill/delete` / `POST /api/cli/skill/undelete`
Soft-delete / restore a skill (owner/admin only).
## Registry discovery (`/.well-known/clawdhub.json`)
The CLI can discover registry/auth settings from the site:
- `/.well-known/clawdhub.json` (JSON)
Schema:
```json
{ "apiBase": "https://clawdhub.com", "authBase": "https://clawdhub.com", "minCliVersion": "0.0.5" }
```
If you self-host, serve this file (or set `CLAWDHUB_REGISTRY` explicitly).
+43
View File
@@ -0,0 +1,43 @@
---
title: 'Mintlify'
description: 'Publish docs/ as a Mintlify docs site.'
---
# Mintlify
Goal: publish `docs/` as a browsable docs site (nice UX for OSS users).
This repo does **not** include Mintlify config yet (`mint.json` missing).
## Minimal setup
1) Install Mintlify CLI (per Mintlify docs).
2) Add a `mint.json` at repo root that points to `docs/` pages.
Example (starter):
```json
{
"name": "ClawdHub",
"logo": "public/logo.svg",
"navigation": [
{ "group": "Start", "pages": ["docs/README", "docs/quickstart"] },
{ "group": "Concepts", "pages": ["docs/architecture", "docs/skill-format", "docs/telemetry"] },
{ "group": "Reference", "pages": ["docs/cli", "docs/http-api", "docs/auth", "docs/deploy"] }
]
}
```
Notes:
- Mintlify usually wants page paths without extension; keep files as `.md`.
- If you prefer Mintlify conventions, rename to `.mdx` later (optional).
## Recommended “docs UX” additions
- Add an “Overview” page (use `docs/README.md`).
- Keep “Quickstart” copy/paste friendly.
- Provide CLI + HTTP API reference pages (done here).
- Add a Troubleshooting page for common setup failures.
+113
View File
@@ -0,0 +1,113 @@
---
title: 'Quickstart'
description: 'Run ClawdHub locally and exercise the CLI + API.'
---
# Quickstart
## 0) Prereqs
- Bun
- Convex CLI (`bunx convex ...`)
- GitHub OAuth App (for login)
- OpenAI key (for embeddings/search)
## 1) Local dev (web + Convex)
```bash
bun install
cp .env.local.example .env.local
# terminal A
bun run dev
# terminal B
bunx convex dev
```
## 2) Auth setup (GitHub OAuth + Convex Auth keys)
Fill in `.env.local`:
- `AUTH_GITHUB_ID`
- `AUTH_GITHUB_SECRET`
- `VITE_CONVEX_URL`
- `VITE_CONVEX_SITE_URL`
- `CONVEX_SITE_URL` (same as `VITE_CONVEX_SITE_URL`)
- `OPENAI_API_KEY`
Generate Convex Auth keys for your deployment:
```bash
bunx auth --deployment-name <deployment> --web-server-url http://localhost:3000
```
Then paste the printed `JWT_PRIVATE_KEY` + `JWKS` into `.env.local` (and ensure the deployment got them too).
## 3) CLI: login + basic commands
From this repo:
```bash
bun clawdhub --help
bun clawdhub login
bun clawdhub whoami
bun clawdhub search gif --limit 5
```
Install a skill into `./skills/<slug>`:
```bash
bun clawdhub install <slug>
bun clawdhub list
```
Update:
```bash
bun clawdhub update --all
```
## 4) Publish a skill
Create a folder containing `SKILL.md` (required) plus any supporting text files:
```bash
mkdir -p /tmp/clawdhub-skill-demo && cd /tmp/clawdhub-skill-demo
cat > SKILL.md <<'EOF'
---
name: Demo Skill
description: Demo skill for local testing
---
# Demo Skill
Hello.
EOF
```
Publish:
```bash
bun clawdhub publish . \
--slug clawdhub-demo-$(date +%s) \
--name "Demo $(date +%s)" \
--version 1.0.0 \
--tags latest \
--changelog "Initial release"
```
## 5) Sync local skills (auto-publish new/changed)
`sync` scans for local skill folders and publishes the ones that arent “synced” yet.
```bash
bun clawdhub sync
```
Dry run + non-interactive:
```bash
bun clawdhub sync --all --dry-run --no-input
```
+57
View File
@@ -0,0 +1,57 @@
---
title: 'Skill Format'
description: 'What the CLI uploads and what the registry stores.'
---
# Skill format
## On disk
A skill is a folder.
Required:
- `SKILL.md` (or `skill.md`)
Optional:
- any supporting *text-based* files (see “Allowed files”)
- `.clawdhubignore` (ignore patterns for publish/sync)
- `.gitignore` (also honored)
Local install metadata (written by the CLI):
- `<skill>/.clawdhub/origin.json`
Workdir install state (written by the CLI):
- `<workdir>/.clawdhub/lock.json`
## `SKILL.md`
- Markdown with optional YAML frontmatter.
- The server extracts metadata from frontmatter during publish.
- `description` is used as the skill summary in the UI/search.
## Allowed files
Only “text-based” files are accepted by publish.
- Extension allowlist is in `packages/schema/src/textFiles.ts` (`TEXT_FILE_EXTENSIONS`).
- Content types starting with `text/` are treated as text; plus a small allowlist (JSON/YAML/TOML/JS/TS/Markdown/SVG).
Limits (server-side):
- Total bundle size: 50MB.
- Embedding text includes `SKILL.md` + up to ~40 non-`.md` files (best-effort cap).
## Slugs
- Derived from folder name by default.
- Must be lowercase and URL-safe: `^[a-z0-9][a-z0-9-]*$`.
## Versioning + tags
- Each publish creates a new version (semver).
- Tags are string pointers to a version; `latest` is commonly used.
+49
View File
@@ -0,0 +1,49 @@
---
title: 'Troubleshooting'
description: 'Common setup and runtime issues (CLI + backend).'
---
# Troubleshooting
## `clawdhub login` opens browser but never completes
- Ensure your browser can reach `http://127.0.0.1:<port>/callback` (local firewalls/VPNs can interfere).
- Use headless mode:
- create a token in the web UI (Settings → API tokens)
- `clawdhub login --token clh_...`
## `whoami` / `publish` returns `Unauthorized` (401)
- Token missing or revoked: check your config file (`CLAWDHUB_CONFIG_PATH` override?).
- Ensure requests include `Authorization: Bearer ...` (CLI does this automatically).
## `publish` fails with `OPENAI_API_KEY is not configured`
- Set `OPENAI_API_KEY` in the Convex environment (not only locally).
- Re-run `bunx convex dev` / `bunx convex deploy` after setting env.
## `sync` says “No skills found”
- `sync` looks for folders containing `SKILL.md` (or `skill.md`).
- It scans:
- workdir first
- then fallback roots (legacy `~/clawdis/skills`, `~/clawdbot/skills`, etc.)
- Provide explicit roots:
```bash
clawdhub sync --root /path/to/skills
```
## `update` refuses due to “local changes (no match)”
- Your local files dont match any published fingerprint.
- Options:
- keep local edits; skip updating
- overwrite: `clawdhub update <slug> --force`
- publish as fork: copy to new folder/slug then `clawdhub publish ... --fork-of upstream@version`
## `GET /api/*` works locally but not on Vercel
- Check `vercel.json` rewrite destination points at your Convex site URL.
- Ensure `VITE_CONVEX_SITE_URL` and `CONVEX_SITE_URL` match your deployment.