Merge pull request #100 from abhi1693/docs/skeleton-45a7c1d

docs: add testing + release checklist; fix contributing links

CONTRIBUTING.md

@@ -11,8 +11,9 @@ This repo welcomes contributions in three broad categories:
 ## Where to start
 
 - Docs landing page: [Docs landing](./docs/README.md)
-- Development workflow: [Development workflow](./docs/03-development.md)
-- Testing guide: [Testing guide](./docs/testing/README.md)
+- Development workflow: [Development](./docs/development/README.md)
+- Testing guide: [Testing](./docs/testing/README.md)
+- Release checklist: [Release checklist](./docs/release/README.md)
 
 ## Filing issues
 

docs/README.md

@@ -1,20 +1,26 @@
 # Mission Control docs
 
-This folder is the starting point for Mission Control documentation.
+This folder is the documentation home for **OpenClaw Mission Control**.
 
-## Sections
+## Start here
 
-- [Development workflow](./03-development.md)
-- [Testing guide](./testing/README.md)
-- [Coverage policy](./coverage-policy.md)
+- [Getting started](./getting-started/README.md)
+- [Development](./development/README.md)
+- [Testing](./testing/README.md)
 - [Deployment](./deployment/README.md)
-- [Production notes](./production/README.md)
+- [Release checklist](./release/README.md)
+- [Operations](./operations/README.md)
 - [Troubleshooting](./troubleshooting/README.md)
 - [Gateway agent provisioning and check-in troubleshooting](./troubleshooting/gateway-agent-provisioning.md)
 - [Gateway WebSocket protocol](./openclaw_gateway_ws.md)
 - [OpenClaw baseline configuration](./openclaw_baseline_config.md)
 
-## Status
+## Reference
 
-These pages are minimal placeholders so repo-relative links stay healthy. The actual docs
-information architecture will be defined in the Docs overhaul tasks.
+- [Configuration reference](./reference/configuration.md)
+- [Authentication](./reference/authentication.md)
+- [API notes](./reference/api.md)
+
+## Contributing to docs
+
+- [Docs style guide](./style-guide.md)

docs/architecture/README.md

@@ -0,0 +1,10 @@
+# Architecture
+
+## High level
+
+- Frontend: Next.js
+- Backend: FastAPI
+- Database: Postgres
+
+> **Note**
+> Add component diagrams and key data flows (auth, task lifecycle, gateway integration) as they solidify.

docs/deployment/README.md

@@ -1,3 +1,100 @@
-# Deployment guide
+# Deployment
 
-Placeholder.
+This section covers deploying Mission Control in self-hosted environments.
+
+> **Goal**
+> A simple, reproducible deploy that preserves the Postgres volume and supports safe upgrades.
+
+## Deployment mode: single host (Docker Compose)
+
+### Prerequisites
+
+- Docker + Docker Compose v2 (`docker compose`)
+- A host where the **browser** can reach the backend URL you configure (see `NEXT_PUBLIC_API_URL` below)
+
+### 1) Configure environment
+
+From repo root:
+
+```bash
+cp .env.example .env
+```
+
+Edit `.env`:
+
+- `AUTH_MODE=local` (default)
+- **Set** `LOCAL_AUTH_TOKEN` to a non-placeholder value (≥ 50 chars)
+- Ensure `NEXT_PUBLIC_API_URL` is reachable from the browser (not a Docker-internal hostname)
+
+Key variables (from `.env.example` / `compose.yml`):
+
+- Frontend: `FRONTEND_PORT` (default `3000`)
+- Backend: `BACKEND_PORT` (default `8000`)
+- Postgres: `POSTGRES_DB`, `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_PORT`
+- Backend:
+  - `DB_AUTO_MIGRATE` (default `true` in compose)
+  - `CORS_ORIGINS` (default `http://localhost:3000`)
+
+### 2) Start the stack
+
+```bash
+docker compose -f compose.yml --env-file .env up -d --build
+```
+
+Open:
+
+- Frontend: `http://localhost:${FRONTEND_PORT:-3000}`
+- Backend health: `http://localhost:${BACKEND_PORT:-8000}/healthz`
+
+### 3) Verify
+
+```bash
+curl -f "http://localhost:${BACKEND_PORT:-8000}/healthz"
+```
+
+If the frontend loads but API calls fail, double-check:
+
+- `NEXT_PUBLIC_API_URL` is set and reachable from the **browser**
+- backend CORS includes the frontend origin (`CORS_ORIGINS`)
+
+## Database persistence
+
+The Compose stack uses a named volume:
+
+- `postgres_data` → `/var/lib/postgresql/data`
+
+This means:
+
+- `docker compose ... down` preserves data
+- `docker compose ... down -v` is **destructive** (deletes the DB volume)
+
+## Migrations / upgrades
+
+### Default behavior in Compose
+
+In `compose.yml`, the backend container defaults:
+
+- `DB_AUTO_MIGRATE=true`
+
+So on startup the backend will attempt to run Alembic migrations automatically.
+
+> **Warning**
+> For zero/near-zero downtime, migrations must be **backward compatible** with the currently running app if you do rolling deploys.
+
+### Safer operator pattern (manual migrations)
+
+If you want more control, set `DB_AUTO_MIGRATE=false` and run migrations explicitly during deploy:
+
+```bash
+cd backend
+uv run alembic upgrade head
+```
+
+## Reverse proxy / TLS
+
+Typical setup (outline):
+
+- Put the frontend behind HTTPS (reverse proxy)
+- Ensure the frontend can reach the backend over the configured `NEXT_PUBLIC_API_URL`
+
+This section is intentionally minimal until we standardize a recommended proxy (Caddy/Nginx/Traefik).

docs/development/README.md

@@ -0,0 +1,59 @@
+# Development
+
+This section is for contributors developing Mission Control locally.
+
+## Recommended workflow (fast loop)
+
+Run Postgres in Docker, run backend + frontend on your host.
+
+### 1) Start Postgres
+
+From repo root:
+
+```bash
+cp .env.example .env
+docker compose -f compose.yml --env-file .env up -d db
+```
+
+### 2) Run the backend (dev)
+
+```bash
+cd backend
+cp .env.example .env
+
+uv sync --extra dev
+uv run uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
+```
+
+Verify:
+
+```bash
+curl -f http://localhost:8000/healthz
+```
+
+### 3) Run the frontend (dev)
+
+```bash
+cd frontend
+cp .env.example .env.local
+npm install
+npm run dev
+```
+
+Open http://localhost:3000.
+
+## Useful repo-root commands
+
+```bash
+make help
+make setup
+make check
+```
+
+- `make setup`: sync backend + frontend deps
+- `make check`: lint + typecheck + tests + build (closest CI parity)
+
+## Related docs
+
+- [Testing](../testing/README.md)
+- [Release checklist](../release/README.md)

docs/getting-started/README.md

@@ -0,0 +1,30 @@
+# Getting started
+
+## What is Mission Control?
+
+Mission Control is the web UI and HTTP API for operating OpenClaw.
+
+It provides a control plane for boards, tasks, agents, approvals, and (optionally) gateway connections.
+
+## Quickstart (Docker Compose)
+
+From repo root:
+
+```bash
+cp .env.example .env
+
+# REQUIRED when AUTH_MODE=local
+# Set LOCAL_AUTH_TOKEN to a non-placeholder value with at least 50 characters.
+
+docker compose -f compose.yml --env-file .env up -d --build
+```
+
+Open:
+- Frontend: http://localhost:3000
+- Backend health: http://localhost:8000/healthz
+
+## Next steps
+
+- [Authentication](../reference/authentication.md)
+- [Deployment](../deployment/README.md)
+- [Development](../development/README.md)

docs/operations/README.md

@@ -0,0 +1,86 @@
+# Operations
+
+Runbooks and operational notes for running Mission Control.
+
+## Health checks
+
+Backend exposes:
+
+- `/healthz` — liveness
+- `/readyz` — readiness
+
+Example:
+
+```bash
+curl -f http://localhost:8000/healthz
+curl -f http://localhost:8000/readyz
+```
+
+## Logs
+
+### Docker Compose
+
+```bash
+# tail everything
+docker compose -f compose.yml --env-file .env logs -f --tail=200
+
+# tail just backend
+docker compose -f compose.yml --env-file .env logs -f --tail=200 backend
+```
+
+The backend supports slow-request logging via `REQUEST_LOG_SLOW_MS`.
+
+## Backups
+
+The DB runs in Postgres (Compose `db` service) and persists to the `postgres_data` named volume.
+
+### Minimal backup (logical)
+
+Example with `pg_dump` (run on the host):
+
+```bash
+# load variables from .env (trusted file only)
+set -a
+. ./.env
+set +a
+
+: "${POSTGRES_DB:?set POSTGRES_DB in .env}"
+: "${POSTGRES_USER:?set POSTGRES_USER in .env}"
+: "${POSTGRES_PORT:?set POSTGRES_PORT in .env}"
+: "${POSTGRES_PASSWORD:?set POSTGRES_PASSWORD in .env (strong, unique value; not \"postgres\")}"
+
+PGPASSWORD="$POSTGRES_PASSWORD" pg_dump \
+  -h 127.0.0.1 -p "$POSTGRES_PORT" -U "$POSTGRES_USER" \
+  -d "$POSTGRES_DB" \
+  --format=custom > mission_control.backup
+```
+
+> **Note**
+> For real production, prefer automated backups + retention + periodic restore drills.
+
+## Upgrades / rollbacks
+
+### Upgrade (Compose)
+
+```bash
+docker compose -f compose.yml --env-file .env up -d --build
+```
+
+### Rollback
+
+Rollback typically means deploying a previous image/commit.
+
+> **Warning**
+> If you applied non-backward-compatible DB migrations, rolling back the app may require restoring the database.
+
+## Common issues
+
+### Frontend loads but API calls fail
+
+- Confirm `NEXT_PUBLIC_API_URL` is set and reachable from the browser.
+- Confirm backend CORS includes the frontend origin (`CORS_ORIGINS`).
+
+### Auth mismatch
+
+- Backend: `AUTH_MODE` (`local` or `clerk`)
+- Frontend: `NEXT_PUBLIC_AUTH_MODE` should match

docs/reference/api.md

@@ -0,0 +1,141 @@
+# API reference (notes + conventions)
+
+Mission Control exposes a JSON HTTP API (FastAPI) under `/api/v1/*`.
+
+- Default backend base URL (local): `http://localhost:8000`
+- Health endpoints:
+  - `GET /health` (liveness)
+  - `GET /healthz` (liveness alias)
+  - `GET /readyz` (readiness)
+
+## OpenAPI / Swagger
+
+- OpenAPI schema: `GET /openapi.json`
+- Swagger UI (FastAPI default): `GET /docs`
+
+> If you are building clients, prefer generating from `openapi.json`.
+
+## API versioning
+
+- Current prefix: `/api/v1`
+- Backwards compatibility is **best-effort** while the project is under active development.
+
+## Authentication
+
+All protected endpoints expect a bearer token:
+
+```http
+Authorization: Bearer <token>
+```
+
+Auth mode is controlled by `AUTH_MODE`:
+
+- `local`: shared bearer token auth (token is `LOCAL_AUTH_TOKEN`)
+- `clerk`: Clerk JWT auth
+
+Notes:
+- The frontend uses the same bearer token scheme in local mode (users paste the token into the UI).
+- Many “agent” endpoints use an agent token header instead (see below).
+
+### Agent auth (Mission Control agents)
+
+Some endpoints are designed for autonomous agents and use an agent token header:
+
+```http
+X-Agent-Token: <agent-token>
+```
+
+In the backend, these are enforced via the “agent auth” context. When in doubt, consult the route’s dependencies (e.g., `require_admin_or_agent`).
+
+## Authorization / permissions model (high level)
+
+The backend distinguishes between:
+
+- **users** (humans) authenticated via `AUTH_MODE`
+- **agents** authenticated via agent tokens
+
+Common patterns:
+
+- **Admin-only** user endpoints: require an authenticated user with admin privileges.
+- **Admin or agent** endpoints: allow either an admin user or an authenticated agent.
+- **Board-scoped access**: user/agent access may be restricted to a specific board.
+
+> SOC2 note: the API produces an audit-friendly request id (see below), but role/permission policy should be documented per endpoint as we stabilize.
+
+## Request IDs
+
+Every response includes an `X-Request-Id` header.
+
+- Clients may supply their own `X-Request-Id`; otherwise the server generates one.
+- Use this id to correlate client reports with server logs.
+
+## Errors
+
+Errors are returned as JSON with a stable top-level shape:
+
+```json
+{
+  "detail": "...",
+  "request_id": "..."
+}
+```
+
+Common status codes:
+
+- `401 Unauthorized`: missing/invalid credentials
+- `403 Forbidden`: authenticated but not allowed
+- `404 Not Found`: resource missing (or not visible)
+- `422 Unprocessable Entity`: request validation error
+- `500 Internal Server Error`: unhandled server errors
+
+Validation errors (`422`) typically return `detail` as a list of structured field errors (FastAPI/Pydantic style).
+
+## Pagination
+
+List endpoints commonly return an `items` array with paging fields (varies by endpoint). If you’re implementing new list endpoints, prefer consistent parameters:
+
+- `limit`
+- `offset`
+
+…and return:
+
+- `items: []`
+- `total`
+- `limit`
+- `offset`
+
+## Examples (curl)
+
+### Health
+
+```bash
+curl -f http://localhost:8000/healthz
+```
+
+### Agent heartbeat check-in
+
+```bash
+curl -s -X POST http://localhost:8000/api/v1/agent/heartbeat \
+  -H "X-Agent-Token: $AUTH_TOKEN" \
+  -H "Content-Type: application/json" \
+  -d '{"name":"Tessa","board_id":"<board-id>","status":"online"}'
+```
+
+### List tasks for a board
+
+```bash
+curl -s "http://localhost:8000/api/v1/agent/boards/<board-id>/tasks?status=inbox&limit=10" \
+  -H "X-Agent-Token: $AUTH_TOKEN"
+```
+
+## Gaps / follow-ups
+
+- Per-endpoint documentation of:
+  - required auth header (`Authorization` vs `X-Agent-Token`)
+  - required role (admin vs member vs agent)
+  - common error responses per endpoint
+- Rate limits are not currently specified in the docs; if enforced, document them here and in OpenAPI.
+- Add canonical examples for:
+  - creating/updating tasks + comments
+  - board memory streaming
+  - approvals workflow

docs/reference/authentication.md

@@ -0,0 +1,30 @@
+# Authentication
+
+Mission Control supports two auth modes via `AUTH_MODE`:
+
+- `local`: shared bearer token auth for self-hosted deployments
+- `clerk`: Clerk JWT auth
+
+## Local mode
+
+Backend:
+
+- `AUTH_MODE=local`
+- `LOCAL_AUTH_TOKEN=<token>`
+
+Frontend:
+
+- `NEXT_PUBLIC_AUTH_MODE=local`
+- Provide the token via the login UI.
+
+## Clerk mode
+
+Backend:
+
+- `AUTH_MODE=clerk`
+- `CLERK_SECRET_KEY=<secret>`
+
+Frontend:
+
+- `NEXT_PUBLIC_AUTH_MODE=clerk`
+- `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=<key>`

docs/reference/configuration.md

@@ -0,0 +1,19 @@
+# Configuration reference
+
+This page collects the most important config values.
+
+## Root `.env` (Compose)
+
+See `.env.example` for defaults and required values.
+
+### `NEXT_PUBLIC_API_URL`
+
+- **Where set:** `.env` (frontend container environment)
+- **Purpose:** Public URL the browser uses to call the backend.
+- **Gotcha:** Must be reachable from the *browser* (host), not a Docker network alias.
+
+### `LOCAL_AUTH_TOKEN`
+
+- **Where set:** `.env` (backend)
+- **When required:** `AUTH_MODE=local`
+- **Policy:** Must be non-placeholder and at least 50 characters.

docs/release/README.md

@@ -0,0 +1,62 @@
+# Release checklist
+
+This is a lightweight, operator-friendly checklist for releasing Mission Control.
+
+> Goal: **no data loss** and **near-zero (ideally zero) user-visible downtime**.
+
+## Before you release
+
+- [ ] Confirm the target version/commit SHA.
+- [ ] Review merged PRs since last release (especially DB schema/auth changes).
+- [ ] Ensure CI is green on the target SHA.
+- [ ] Confirm you have:
+  - [ ] access to the host(s)
+  - [ ] access to Postgres backups (or snapshots)
+  - [ ] a rollback plan
+
+## Database safety
+
+- [ ] Verify migrations are **backward compatible** with the current running app (if doing rolling deploys).
+- [ ] Take a backup / snapshot.
+- [ ] If migrations are risky or not backward compatible, schedule a maintenance window.
+
+## Deploy (Docker Compose)
+
+- [ ] Pull / build the new images (or update the repo checkout).
+- [ ] Apply migrations (if you run them manually):
+
+```bash
+# example: if running backend locally on the host
+cd backend
+uv run alembic upgrade head
+```
+
+- [ ] Restart services with minimal disruption:
+
+```bash
+docker compose -f compose.yml --env-file .env up -d --build
+```
+
+## Post-deploy verification
+
+- [ ] Backend health: `GET /healthz` returns 200
+- [ ] Backend readiness: `GET /readyz` returns 200
+- [ ] Frontend loads (no console spam)
+- [ ] Login works (local/clerk mode)
+- [ ] Core flows work end-to-end:
+  - [ ] View board
+  - [ ] Create/update a task
+  - [ ] Post a comment
+  - [ ] Heartbeat check-in succeeds
+
+## Rollback (if needed)
+
+- [ ] Roll back the app version (compose / images).
+- [ ] If migrations were applied and are not reversible, rollbacks may require a DB restore.
+
+## Notes to keep this honest
+
+- If you add a new operational dependency (e.g., redis), update:
+  - `README.md` (overview + quickstart)
+  - `docs/deployment/README.md`
+  - this checklist

docs/style-guide.md

@@ -0,0 +1,39 @@
+# Docs style guide
+
+## Principles
+
+- **Be concrete.** Prefer commands, examples, and “expected output” over prose.
+- **Don’t invent behavior.** If unsure, link to the source file and mark it as “verify”.
+- **Optimize for scanning.** Short sections, bullets, and tables.
+- **Call out risk.** Anything destructive or security-sensitive should be labeled clearly.
+
+## Markdown conventions
+
+- Use sentence-case headings.
+- Prefer fenced code blocks with a language (`bash`, `yaml`, `json`).
+- For warnings/notes, use simple callouts:
+
+```md
+> **Note**
+> ...
+
+> **Warning**
+> ...
+```
+
+## Common templates
+
+### Procedure
+
+1. Prereqs
+2. Steps
+3. Verify
+4. Troubleshooting
+
+### Config reference entry
+
+- **Name**
+- **Where set** (`.env`, env var, compose)
+- **Default**
+- **Example**
+- **Notes / pitfalls**

docs/testing/README.md

@@ -1,3 +1,82 @@
-# Testing guide
+# Testing
 
-Placeholder: see root `README.md` and `CONTRIBUTING.md` for current commands.
+This guide describes how to run Mission Control tests locally.
+
+## Quick start (repo root)
+
+```bash
+make setup
+make check
+```
+
+`make check` is the closest thing to “CI parity”:
+
+- backend: lint + typecheck + unit tests (with scoped coverage gate)
+- frontend: lint + typecheck + unit tests (Vitest) + production build
+
+## Backend tests
+
+From repo root:
+
+```bash
+make backend-test
+make backend-coverage
+```
+
+Or from `backend/`:
+
+```bash
+cd backend
+uv run pytest
+```
+
+Notes:
+
+- Some tests may require a running Postgres (see root `compose.yml`).
+- `make backend-coverage` enforces a strict coverage gate on a scoped set of modules.
+
+## Frontend tests
+
+From repo root:
+
+```bash
+make frontend-test
+```
+
+Or from `frontend/`:
+
+```bash
+cd frontend
+npm run test
+npm run test:watch
+```
+
+## End-to-end (Cypress)
+
+The frontend has Cypress configured in `frontend/cypress/`.
+
+Typical flow:
+
+1) Start the stack (or start backend + frontend separately)
+2) Run Cypress
+
+Example (two terminals):
+
+```bash
+# terminal 1
+cp .env.example .env
+docker compose -f compose.yml --env-file .env up -d --build
+```
+
+```bash
+# terminal 2
+cd frontend
+npm run e2e
+```
+
+Or run interactively:
+
+```bash
+cd frontend
+npm run e2e:open
+```

docs/troubleshooting/README.md

@@ -1,3 +1,12 @@
 # Troubleshooting
 
 - [Gateway agent provisioning and check-in](./gateway-agent-provisioning.md)
+
+## Common issues
+
+- Frontend can’t reach backend (check `NEXT_PUBLIC_API_URL`)
+- Auth errors (check `AUTH_MODE`, tokens)
+- DB connection/migrations
+
+> **Note**
+> Expand with concrete symptoms + fixes as issues are discovered.