admin/OpenJarvis
1
0
Fork 0
mirror of https://github.com/open-jarvis/OpenJarvis.git synced 2026-08-14 08:52:06 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(mining): Pearl mining integration

Browse Source 
Consolidates NVIDIA vLLM, Apple Silicon, CPU Pearl mining support, CLI/docs, and live H100 validation.
This commit is contained in:
Jon Saad-Falcon
2026-05-05 19:11:15 -07:00
committed by GitHub
parent e79bc1e196
commit f41cf420be
74 changed files with 14533 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/design/2026-05-05-apple-silicon-pearl-mining-design.md
+711
View File
@@ -0,0 +1,711 @@
# Spec B — Apple Silicon enablement for Pearl mining
| | |
|---|---|
| **Date** | 2026-05-05 |
| **Status** | Design — Phase 0 investigation complete; v1 scope reduced (see §1.5) |
| **Owner** | OpenJarvis team (parallel-agent friendly) |
| **Companion spec** | [Spec A — vLLM-Pearl mining integration (v1)](2026-05-05-vllm-pearl-mining-integration-design.md) |
| **Repos referenced** | `OpenJarvis`, `pearl-research-labs/pearl`, possibly upstream `ml-explore/mlx`, `ggerganov/llama.cpp` |
> **For agents picking this up cold:** read §1 ("Cold-start brief") first, then **§1.5 ("Phase 0 findings")** which substantially simplifies v1 scope. The original Phase 1–4 GPU-kernel plan in §5–§8 is preserved as the v2/v3 path; v1 ships using upstream Pearl's PyTorch reference and pure-Rust miner.
## 1. Cold-start brief
**One-paragraph problem statement.** OpenJarvis is adding a `mining` subsystem (Spec A) that lets users mine the Pearl PoUW blockchain through their local LLM inference. Pearl's reference miner is CUDA-only and bound to NVIDIA Hopper (`sm_90a`, H100/H200) — most OpenJarvis users on Apple Silicon are locked out at the protocol level. **This spec is the plan to unblock them.** The OJ-side integration work is small (a new `MiningProvider` implementation that drops into the existing `MinerRegistry` from Spec A); the substantial work is a Metal port of Pearl's `NoisyGEMM` kernel and a matching plugin into an Apple-native inference backend (MLX or llama.cpp Metal). The Pearl validation path is plonky2-STARK-based and **already hardware-neutral** — see §3 for the evidence — so a correct Metal implementation produces blocks Pearl validators accept without any consensus changes.
**Three things to know before doing any work.**
1. The Pearl validator (`pearl/zk-pow/src/api/verify.rs::verify_block`) operates on a STARK proof and references no hardware. **The protocol does not care what GPU produced the work** as long as the math is correct and the proof verifies. CUDA-only is a performance choice, not a consensus choice.
2. Pearl's CUDA kernel (`pearl/miner/pearl-gemm/csrc/gemm/`) uses Hopper-only primitives (TMA, WGMMA, thread-block clusters, CUTLASS 3.x). A Metal port is **not a translation** — it's a from-scratch reimplementation against a different programming model. Plan effort accordingly.
3. The OJ integration boundary is the `MiningProvider` ABC defined in Spec A §4.4. **Do not modify Spec A.** Add a new provider file (`mining/mlx_pearl.py` or `mining/llamacpp_pearl_metal.py`), implement the ABC, register via `MinerRegistry`, ship a new optional extra. Everything else in Spec A — sidecar shape, config schema, telemetry adapter contract, v2 fee/pool seams — applies unchanged.
## 1.5. Phase 0 findings — significant scope simplification (2026-05-05)
Phase 0 investigation produced four findings that reshape this spec. **The original §5–§8 plan (Metal NoisyGEMM kernel + custom MLX/llama.cpp plugin) is preserved as v2/v3, but is no longer required for v1.**
### 1.5.1 The validator is hardware-neutral (confirmed)
`zk-pow/src/api/verify.rs::verify_block` and `verify_plain_proof` are pure Rust + plonky2; no CUDA, no GPU paths, no hardware introspection. §3.1's claim is **verified by direct code reading** (see file paths in §11). The protocol accepts blocks from any implementation that produces correct math.
### 1.5.2 A complete hardware-neutral miner already exists upstream
`pearl/zk-pow/src/ffi/mine.rs::mine()` is a pure-Rust mining function:
- Generates random `i8` matrices `A` (m×k) and `B` (k×n) with values in `[-64, 64]`
- Computes blake3-derived noise via `circuit/pearl_noise.rs::compute_noise_for_indices`
- Performs the noised dot products in tile patterns (`PeriodicPattern.rows_pattern × cols_pattern`)
- Hashes the jackpot tile and checks the difficulty target
- Returns a `PlainProof` that `verify_plain_proof` accepts
It is exposed to Python via `py-pearl-mining` (`pearl_mining.mine`). Dependencies: pure Rust (`zk-pow`, `pearl-blake3`, `blake3`, `rayon`, `pyo3`, `tikv-jemallocator`). **No CUDA, no platform-specific code in the Cargo dependency tree.**
### 1.5.3 A PyTorch reference of the *production* NoisyGEMM also exists upstream
`pearl/miner/miner-base/src/miner_base/noisy_gemm.py::NoisyGemm` is a complete PyTorch reference of the same NoisyGEMM that vllm-miner accelerates with the H100 CUDA kernel:
- `noise_A`, `noise_B`, `gemm`, `noisy_gemm` methods replicating the kernel's math in `torch.matmul` calls
- The denoising path produces **bit-exact results** versus a vanilla `torch.matmul(A.int32, B.int32)` — verified by the `assert torch.equal(result, expected)` test at `miner/miner-base/tests/test_noisy_gemm.py:92`. The "fp16 tolerance" budget I assumed in the original §5.2 is unnecessary for the int7×int7→int32 protocol path
- Dependencies: `torch==2.11.0`, `blake3`, `numpy`, `pearl-gateway`, `py-pearl-mining` — all install on macOS arm64
### 1.5.4 Empirical confirmation (2026-05-05, on this hardware)
`py-pearl-mining` was built from the upstream source on the spec author's M2 Max. The build produced `py_pearl_mining-0.1.0-cp312-abi3-macosx_11_0_arm64.whl` in 56 seconds. End-to-end mining cycle:
```
running mine(m=256, n=128, k=1024, rank=32) on Apple Silicon CPU…
mine() returned a proof in 0.078s
verify_plain_proof: ok=True, msg='Mining solution verified successfully' (0.3 ms)
END-TO-END MINING ON APPLE SILICON SUCCEEDED
```
The test difficulty here is `nbits=0x1D2FFFFF` (the test fixture from `py-pearl-mining/tests/test_python_api.py`), much lower than mainnet difficulty — so 78 ms is **per-share at the test difficulty**, not real expected wall-clock per share at the network's current difficulty. But the *correctness* of the path is proven.
### 1.5.5 Reframed v1 — what to build, what to defer
| Layer | Original plan (§5–§8) | New v1 plan |
|---|---|---|
| Reference oracle (Phase 0-B) | Build from scratch in PyTorch, validate against H100 CUDA | **Already exists upstream** — `miner-base.noisy_gemm` + `pearl_mining.mine`. OJ ships a thin wrapper, no reimplementation. |
| Inference-backend plugin (Phase 2) | Custom MLX or llama.cpp Metal plugin doing NoisyGEMM | **Deferred to v2.** v1 is **decoupled mining** — mining runs as a separate process via the upstream Rust miner; user's existing inference (Ollama, MLX, llama.cpp) is unaffected. |
| Metal NoisyGEMM kernel (Phase 1) | Months of GPU-kernel engineering | **Deferred to v3** as a perf optimization once v1 ships and demand is proven. Original §6.1 content preserved as the v3 plan. |
| OJ provider integration (Phase 3) | New `MiningProvider` impl | **v1 ships this** — see §13. `MiningProvider` ABC from Spec A unchanged. |
| Bringup + verification (Phase 4) | Hardware matrix + testnet | **v1 ships this** — see §14. Same hardware matrix, simpler scope. |
| Pearl coordination (Phase 0-A) | Confirm upstream-vs-fork posture | Still needed (see §12) — but the bar is lower since v1 doesn't require any code from us in Pearl's tree. |
### 1.5.6 Honest performance expectations for v1
This is **not** competitive mining. The point of vllm-miner is that it amortizes mining work over LLM inference matmuls (the matmul you're already doing for inference *is* the mining work). v1 here decouples them: your CPU does mining, your GPU does inference. The hashrate will be low. **But it works today, and it ships with a credible upgrade path.** Document this transparently in `mine doctor` and the user guide.
The v2 (PyTorch-MPS NoisyGEMM coupled with MLX/llama.cpp inference) and v3 (native Metal kernel) work paths in §5–§8 remain the route to competitive Apple Silicon mining. They're explicitly not blocking v1.
## 2. Why this is its own spec
Spec A's scope is the v1 integration that ships today on the only working configuration (vLLM + sm90). Apple Silicon enablement is a separate, parallelizable workstream because:
- **Different ownership boundary.** Spec A is Python integration of an existing Pearl Docker image. Spec B is GPU-kernel engineering with potential upstream contribution to Pearl. These need different reviewers, different CI surfaces (no H100 needed, but Apple Silicon required), and different release cadence.
- **Different timeline.** Spec A is weeks. Spec B is plausibly months for the kernel work alone.
- **Different blast radius.** Spec A ships zero risk to non-mining users; even mining users who edit the wrong config get a clear error. Spec B carries protocol-correctness risk — a bug in NoisyGEMM produces invalid blocks that get rejected by validators.
- **Parallel-agent ergonomics.** The user has explicitly asked for this spec to be picked up by a separate agent in parallel. Self-containment is a design goal.
## 3. Evidence: Apple Silicon support is possible
### 3.1 The validator is hardware-neutral
`pearl/zk-pow/src/api/verify.rs::verify_block`:
```rust
pub fn verify_block(public_params: &PublicProofParams, proof: &ZKProof, cache: &mut CircuitCache) -> Result<()> {
let (params, pis) = prepare_verification(public_params, proof, None)?;
PearlRecursion::compile_circuits(params, cache, false)?;
verify_with_cache(params, cache, &pis, proof)
}
```
Verification is `PearlRecursion::verify(params, cache, pis, &proof.plonky2_proof)` — a recursive plonky2 STARK check. No GPU code path, no CUDA dependency. Validator nodes run pure Rust.
The mining work consists of three things, all of which are mathematical specifications — not implementation specifications:
1. A NoisyGEMM result whose noise pattern is derived from blake3 of a per-block key
2. A blake3 commitment hash over the noised matmul that meets a difficulty target
3. A plonky2 STARK proof that ties the result to the commitment
Any implementation that produces matching outputs is acceptable to the network. **This is the design intent of PoUW** — the work has to be replayable and verifiable, but not hardware-bound.
### 3.2 The Pearl team explicitly anticipates non-CUDA plugins
From `pearl/miner/README.md`:
> "Currently only mining via vLLM is supported, in the future we hope to supply plugins for other LLM inference libraries, like SGLang, TensorRT-LLM, Ollama, ..."
Apple is not in their list, but the framing — "supply plugins for other LLM inference libraries" — implies the boundary is at the inference backend, not at the consensus protocol. Confirms the architectural read.
### 3.3 Reference implementation exists in py-pearl-mining
`pearl/py-pearl-mining/` is a PyO3 crate exposing Pearl mining primitives in Python. **Read it before designing the Metal port** — it likely contains the protocol-relevant constants in a hardware-neutral form, suitable as a reference oracle for Phase 0 testing (§5).
## 4. Scope
### 4.1 In scope
- **Phase 0** (§5): protocol-acceptance verification + Pearl-team coordination + Python reference oracle
- **Phase 1** (§6.1): Metal NoisyGEMM kernel — the substantive engineering
- **Phase 2** (§6.2): inference-backend plugin — MLX or llama.cpp Metal
- **Phase 3** (§7): OJ provider integration — new `MiningProvider` impl, new optional extra, registry hookup
- **Phase 4** (§8): verification matrix across Apple Silicon variants and bringup on Pearl testnet
- Documentation deliverables and the upstream-contribution path
### 4.2 Out of scope
- Pool support and the 20% OJ fee (Spec A §8.5; that lives in a future v2 pool spec)
- Custody, signing, or routing Pearl funds (Spec A anti-goal; same here)
- AMD ROCm enablement (separate spec, parallel structure to this one)
- Intel Arc / Mac Intel / older CUDA enablement (separate specs)
- Modifying anything in Spec A. **Spec B is purely additive.**
- Pearl protocol changes (none required; see §3.1)
### 4.3 Explicit non-goal: economic competitiveness
This spec does not promise that Apple Silicon mining will be **profitable**. The performance gap to a tuned H100 kernel is likely large (§6.1.5 discusses why). What this spec *does* promise: a correct, working Apple Silicon path that's enabled the day the kernel ships, with a transparent doctor surface that tells Mac users honestly what their hashrate looks like. Whether it's worth the electricity is a user decision.
## 5. Phase 0 — investigation, coordination, reference oracle
The phase that costs the least and prevents the most rework. Three workstreams in parallel.
### 5.1 Workstream P0-A: Pearl-side coordination
**Goal:** Confirm protocol acceptance in writing from Pearl maintainers; align on whether OJ contributes upstream or ships independently.
**Steps:**
1. Open a GitHub Discussion on `pearl-research-labs/pearl`: "Apple Silicon / Metal NoisyGEMM enablement — coordination". Reference Spec B URL.
2. Get explicit confirmation from a Pearl maintainer that:
- Validator path is hardware-neutral as believed (§3.1).
- There is no Pearl-internal Metal port already in flight that would conflict.
- LICENSE compatibility allows OJ-authored kernel code to be either contributed upstream (preferred) or distributed alongside OJ.
3. Discuss the upstream-vs-fork question. Strong default: **contribute upstream into a new `pearl/miner/pearl-gemm-metal/` crate**, paralleling `pearl-gemm/`, so Pearl owns the kernel long-term and we benefit from their CI and review. Fork only if upstream contribution is blocked.
**Exit criteria:**
- [ ] Written confirmation of protocol acceptance
- [ ] Agreement on contribution model (upstream / coordinated fork / independent)
- [ ] No duplicate-effort risk
### 5.2 Workstream P0-B: build a reference oracle
**Goal:** A pure-Python (or pure-Rust) implementation of NoisyGEMM that produces bit-exact-or-fp16-tolerance-bounded outputs versus Pearl's CUDA reference. **Used as the test oracle for Phase 1** — without it, you can't verify the Metal kernel's correctness against a portable baseline.
**Steps:**
1. Read `pearl/miner/pearl-gemm/csrc/gemm/` end-to-end. Catalog the protocol-relevant constants in `pearl_gemm_constants.hpp`:
- `kAxEBLScaleFactor = 1 << 14`
- `kEARxBpEBScaleFactor = 1 << 12`
- `kIntToFp16ScaleFactor = 1 << 12`
- `kEBRScaleFactorDenoise`, `kEALScaleFactorDenoise`
2. Read `pearl/py-pearl-mining/` to see what's already exposed in Python. If a reference impl already lives there, **use it**; do not duplicate.
3. If gaps exist, build them in PyTorch (CPU). Mirror the structure of the CUDA kernels:
- `noise_generation.cu` → `noise_generation.py` — derive `EAL`, `EAR`, `EBL`, `EBR` from blake3-of-key + seed
- `pearl_gemm` (matmul + noise) → `pearl_gemm.py` — compute `Y_noisy = (A + EAL·EAR) × (B + EBL·EBR)` with the documented scaling
- `inner_hash_kernel.cu` → `inner_hash.py` — blake3 commitment over the noised matmul
- `denoise_converter.cu` → `denoise.py` — recover `Y_clean = A·B` from `Y_noisy` and the noise components
- `pow_utils.hpp` → `pow_check.py` — difficulty target check
4. Cross-check: run a corpus of 100+ inputs through the Pearl CUDA reference (on an H100 dev box; see §5.4) and through the Python reference. Assert outputs match within the documented tolerance — most likely **bit-exact for int paths and fp16-tolerance for the denoised result**.
**Exit criteria:**
- [ ] `tools/pearl-reference-oracle/` (in OJ repo, or separate repo) builds and tests pass
- [ ] Parity confirmed against Pearl CUDA on ≥100 input sets
- [ ] Constants table documented in this spec (replace the bullet list above with the verified values)
### 5.3 Workstream P0-C: Apple-side viability
**Goal:** Decide between MLX and llama.cpp Metal as the integration host before designing the kernel.
**Decision criteria:**
| Factor | MLX (`ml-explore/mlx`, `mlx-lm`) | llama.cpp Metal (`ggerganov/llama.cpp`) |
|---|---|---|
| Op-replacement hooks | Less mature; would likely require monkey-patching `mlx.nn.Linear` or upstream PR adding plugin hooks | More mature; `ggml` op tree is open and Metal backend has clear extension points (`ggml-metal.metal`) |
| Apple-native quantization story | Excellent (4-bit, 8-bit native ops) | Good but not as native |
| Inference-quality fidelity for OJ users today | High — MLX-LM is the de facto Mac LLM stack | High — also widely used |
| Upstream-contribution complexity | Higher (smaller team, less plugin culture) | Lower (large open community, clear contributor flow) |
| Ecosystem alignment with OJ engine map | OJ's `engine/` doesn't currently have an MLX engine; would need both | OJ already has llama.cpp via `engine/openai_compat_engines.py` |
**Recommendation:** **llama.cpp Metal first**, MLX as a fast-follow. Reasoning: ggml's op tree gives a cleaner extension path for a custom NoisyGEMM op; OJ already has llama.cpp engine wiring; and the upstream-contribution path is more navigable. MLX is a better long-term fit for Apple-native users but is currently a harder integration target.
**Steps:**
1. Spike: implement a no-op "custom op" passthrough in llama.cpp Metal. ~1-2 days work to confirm the integration mechanism is real and the build pipeline cooperates.
2. Spike: same in MLX. Compare effort.
3. Pick one. Document the decision in this spec.
**Exit criteria:**
- [ ] Decision made and documented in §6.2
- [ ] Trivial plugin hook proven on the chosen backend
### 5.4 Hardware required for Phase 0
- One H100/H200 box (cloud rental fine — Lambda, RunPod, Crusoe). Used for: running Pearl's CUDA reference to capture parity test vectors, running the Pearl Docker miner end-to-end as a known-good baseline.
- Apple Silicon dev machines: M2 Max minimum, M3/M4 Pro+ preferred. M-series Ultra ideal for any perf experiments.
- Estimated cloud cost for Phase 0: < $200.
## 6. Phases 1 and 2 — kernel and plugin
### 6.1 Phase 1 — Metal NoisyGEMM kernel
**Goal:** A Metal compute-shader implementation of NoisyGEMM that produces outputs matching the Phase 0 reference oracle, performant enough to make Mac mining a real (if low-yield) feature.
#### 6.1.1 Implementation surface
Two viable targets, in order of preference:
**A. Direct Metal Shading Language (MSL) compute kernels.** Maximum control, maximum performance ceiling, maximum effort. The CUDA reference is highly tuned (TMA, WGMMA, multi-stage pipelines); a direct MSL port can lean on Apple's matmul intrinsics where they exist (`simdgroup_matrix` ops on M3+).
**B. Metal Performance Shaders Graph (MPSGraph).** Higher-level than raw MSL; uses Apple's tuned matmul kernels under the hood; limited control over the in-kernel commitment hash. Likely path: do the matmul via MPSGraph, do noise generation + commitment hashing as separate kernels, accept the perf hit from less fusion.
**Recommendation:** Start with B for correctness and shipping speed; profile; move hot paths to A only if economically justified. Apple's matmul intrinsics are fast enough that the perf gap to a fused implementation may be acceptable.
#### 6.1.2 Algorithm structure
Following the Pearl CUDA reference, end-to-end work performed for one mining attempt:
1. **Quantize inputs.** `A: fp16 → int8 + scale_A`, `B: fp16 → int8 + scale_B`. Match Pearl's `quantize_kernel.cu` semantics (per-row or per-channel scales — verify via Phase 0 oracle).
2. **Generate noise tensors.** From `key_A`, `key_B` (per-block blake3-derived seeds), produce `EAL` (m, R), `EAR` (k, R), `EBL` (k, R), `EBR` (n, R) of int8. Scale factors per `pearl_gemm_constants.hpp`.
3. **Noisy matmul.** Compute `Y_noisy = (A + EAL · EAR_T) × (B + EBL · EBR_T)`. Output is int32 then converted to fp16.
4. **Inner-hash commitment.** blake3 over `Y_noisy` (or a row-tile of it) to produce the PoW target candidate. This is the hottest path — the noise + commitment loop runs at every share.
5. **PoW check.** Compare commitment digest against the difficulty target (`make_pow_target_tensor` semantics from Pearl's Python interface).
6. **On hit: denoise.** Compute `Y_clean = Y_noisy - (noise contributions)` to feed back into vLLM/MLX as the actual matmul output. Inference cannot be wrong.
7. **Post-hit: STARK proof generation.** When a share meets the network difficulty target, the miner generates a plonky2 STARK proof tying the noisy matmul + commitment to the block. This proving step is **separate from the Metal kernel** — it runs in pure Rust via Pearl's existing `zk-pow/` and `py-pearl-mining` code paths and should work cross-platform unchanged. Cost: seconds-to-minutes of CPU per block. Confirm cross-platform builds during Phase 0-C and §7.5.
#### 6.1.3 Crate / package layout
Strong preference: **upstream contribution to Pearl** as `pearl/miner/pearl-gemm-metal/` paralleling the existing `pearl-gemm/`:
```
pearl/miner/pearl-gemm-metal/
Cargo.toml (or pyproject.toml + setup.py — match Pearl conventions)
metal/ (.metal MSL source files)
src/
lib.rs (or src/pearl_gemm_metal/__init__.py)
tests/
```
If upstream contribution is blocked (Phase 0 outcome), fork with attribution into `OpenJarvis/vendor/pearl-gemm-metal/` and document the divergence policy in this spec.
#### 6.1.4 Testing
- **Parity tests.** Each kernel (noise gen, matmul, inner hash, denoise, PoW check) tested independently against the Phase 0 reference oracle. Bit-exact for int paths; fp16-tolerance bounded for fp paths (specific tolerance: TBD via Phase 0 measurement).
- **End-to-end correctness.** Full mining attempt produces a candidate proof that the reference Rust prover (`zk-pow/`) accepts.
- **Hardware fuzz.** Run on M1 Pro, M2 Max, M3 Max, M4 Max, and M-Ultra variants. Catch any silently-wrong hardware behavior (Metal feature variance across generations is real).
#### 6.1.5 Performance expectations
Honest baseline: **expect 0.05–0.2× the share rate of an H100** on a high-end M-Ultra, and proportionally less on smaller chips. Reasons:
- H100 has dedicated FP8/FP16 tensor cores with WGMMA throughput Apple Silicon does not match
- Pearl's CUDA kernel is heavily fused (matmul + noise + commitment in one kernel via TMA pipelining); a Metal version will likely be less fused
- 70B model bandwidth requirements stress unified memory
This is fine. Mac mining is a feature for Apple Silicon owners who want to participate, not a competitive yield product. Document it transparently in `mine doctor` and the user guide.
#### 6.1.6 Exit criteria for Phase 1
- [ ] Parity tests pass on M2 Max and M4 Max
- [ ] End-to-end mining attempt produces a valid proof accepted by `zk-pow::verify_block`
- [ ] Performance characterized and published (M-series matrix)
- [ ] Code merged upstream OR forked-with-policy per Phase 0 outcome
### 6.2 Phase 2 — Inference-backend plugin
**Goal:** A llama.cpp Metal (or MLX, per Phase 0-C) plugin that swaps the standard quantized linear op for Phase 1's NoisyGEMM during inference, so a Mac running this plugin produces both correct LLM outputs and valid mining shares.
#### 6.2.1 Path: llama.cpp Metal (assuming Phase 0-C selected this)
- Add a custom `ggml` op `GGML_OP_PEARL_NOISY_GEMM` with a Metal backend implementation that calls Phase 1's kernels.
- Plugin entry point: a small library that, when loaded, replaces the default linear op in the model graph during loading.
- Build artifact: `libpearl_metal_plugin.dylib` (or static lib).
#### 6.2.2 Path: MLX (alternate)
- Define `mlx.NoisyLinear` as a subclass of `mlx.nn.Linear` that calls Phase 1's kernels via a custom Metal op binding.
- Provide a model-loading shim: `from openjarvis.mining import patch_mlx_for_pearl; patch_mlx_for_pearl()` that monkey-patches `mlx.nn.Linear` instances at load time. Less elegant; works.
#### 6.2.3 Inference-quality regression tests
The plugin is correctness-critical: a noised model that doesn't fully denoise produces degraded responses. Test:
- Load a small reference model (e.g., a 1-3B parameter Pearl-blessed model if one exists for testing, otherwise the smallest model the protocol accepts).
- Run a fixed prompt set through both noised+denoised and standard paths.
- Assert outputs are bit-exact or within fp16 tolerance.
- Run OJ's existing eval framework (`src/openjarvis/evals/`) on a small benchmark (e.g., an MMLU subset registered as a Pearl-mining-mode dataset). Assert no degradation > the tolerance budget. Falling back to `lm-eval-harness` is acceptable if OJ's eval surface for Mac is incomplete at the time.
#### 6.2.4 Exit criteria
- [ ] Plugin loads in chosen backend
- [ ] End-to-end inference produces correct outputs (regression tests pass)
- [ ] Mining shares are submitted to a Pearl testnet during inference
- [ ] At least one block found on testnet from a Mac
## 7. Phase 3 — OpenJarvis provider integration
Where the OJ-side work is small. Inherits the entire `MiningProvider` ABC, registry, sidecar, config schema, telemetry adapter, and v2 seams from Spec A unchanged.
### 7.1 New files
```
src/openjarvis/mining/
llamacpp_pearl_metal.py # OR mlx_pearl.py — depending on Phase 2 path
# @MinerRegistry.register("llamacpp-pearl-metal")
# implements MiningProvider ABC from Spec A §4.4
```
### 7.2 New optional extra
```toml
# pyproject.toml
mining-pearl-metal = [
"pearl-metal-plugin>=0.1", # the Phase 2 plugin, however published
# MLX path adds: "mlx>=0.X", "mlx-lm>=0.X"
# llama.cpp path adds: "llama-cpp-python>=0.X" with Metal extras
]
```
### 7.3 Capability detection
```python
# src/openjarvis/mining/llamacpp_pearl_metal.py
class LlamaCppPearlMetalProvider(MiningProvider):
provider_id = "llamacpp-pearl-metal"
@classmethod
def detect(cls, hw: HardwareInfo, engine_id: str, model: str) -> MiningCapabilities:
if hw.platform != "darwin":
return MiningCapabilities(False, reason="Apple Silicon required (platform != darwin)")
if hw.gpu is None or hw.gpu.vendor != "apple":
return MiningCapabilities(False, reason="Apple Silicon GPU required")
if engine_id not in {"llamacpp", "llama-cpp"}:
return MiningCapabilities(False, reason=f"engine '{engine_id}' has no Pearl Metal plugin; use llamacpp")
if not _pearl_metal_plugin_available():
return MiningCapabilities(False, reason="install with `uv sync --extra mining-pearl-metal`")
if not _model_has_pearl_variant(model):
return MiningCapabilities(False, reason=f"model '{model}' has no Pearl-blessed variant")
return MiningCapabilities(True, estimated_hashrate=_estimate_hashrate(hw))
```
Each branch is exactly the kind of "why can't I mine" message Spec A's `mine doctor` surfaces verbatim.
### 7.4 Lifecycle
Unlike Spec A's vLLM provider which orchestrates a Docker container, the Apple provider runs **two coordinated subprocesses directly on the host**:
1. The inference server (llama.cpp server with the Pearl Metal plugin loaded, or MLX-LM server depending on Phase 0-C path)
2. `pearl-gateway` as a sibling process — same one that runs inside the Docker container in Spec A, but here it runs natively on the Mac
Lifecycle:
- `start()`: spawn (1) with the Pearl Metal plugin pre-loaded (`DYLD_INSERT_LIBRARIES`-style or `--plugin` flag depending on chosen backend's invocation contract), then spawn (2) pointing at it. Write the same sidecar shape Spec A defines, with `gateway_url` pointing at the native pearl-gateway. Track both PIDs internally.
- `stop()`: SIGTERM (2) first, then (1), with bounded waits and SIGKILL fallback. Remove sidecar.
- `is_running()`, `stats()`: identical contract to vLLM provider; `stats()` reads from the native pearl-gateway's `:8339/metrics`.
**No Docker.** Apple Silicon Docker doesn't pass through Metal; running Pearl in a Mac Docker container would defeat the purpose. Document this explicitly in §7 of this spec; do not attempt a Docker path.
### 7.5 Pearl gateway on Mac
The Pearl `pearl-gateway` process is currently only documented as part of the Docker container. For Mac, we need it to run natively. Two options:
1. Build `pearl-gateway` from source via `uv sync --package pearl-gateway` — same workspace package the Docker image uses. Should work cross-platform since it's pure Python plus py-pearl-mining bindings. Verify.
2. If (1) fails on Apple Silicon, work with Pearl maintainers (Phase 0-A) to port it — a small amount of work compared to the kernel.
**Phase 3 verifies (1).** This is a Phase 0-A coordination point.
### 7.6 Exit criteria
- [ ] `LlamaCppPearlMetalProvider` registered, detection matrix correct on M1/M2/M3/M4
- [ ] `jarvis mine init` runs to completion on Apple Silicon
- [ ] `jarvis mine start` launches subprocess + Pearl gateway on Mac
- [ ] `jarvis mine status` returns valid `MiningStats` from a real Mac mining session
- [ ] `jarvis mine doctor` produces honest, actionable output for Mac users
## 8. Phase 4 — Verification & bringup
### 8.1 Hardware matrix
| Chip | Test priority | Expected outcome |
|---|---|---|
| M1 / M1 Pro / M1 Max | low — generation 1 GPU may have feature gaps | works but slow |
| M2 / M2 Pro / M2 Max | medium | works |
| M2 Ultra | medium | best M2-class hashrate |
| M3 / M3 Pro / M3 Max | high — first gen with `simdgroup_matrix` | works, meaningful share rate |
| M4 / M4 Pro / M4 Max | high — current flagship | best non-M-Ultra hashrate |
For each chip in the matrix, run:
1. `jarvis mine init` end-to-end
2. `jarvis mine start` and run for ≥4 h continuous
3. Capture and publish: shares submitted, shares accepted, block-find time distribution, GPU temp, system load impact on normal use
4. Run a parallel `lm-eval-harness` on the mining endpoint to assert inference quality is unaffected
### 8.2 Pearl testnet bringup
Before any mainnet recommendation:
- Mine on Pearl testnet for ≥7 continuous days from at least two Apple Silicon variants
- Find at least one block on testnet from each variant
- Verify all blocks accepted by `zk-pow::verify_block` on a reference validator node
- Report results to Pearl maintainers; gate any mainnet announcement on their sign-off
### 8.3 Documentation deliverables
- `docs/user-guide/mining-apple-silicon.md` — user-facing: prerequisites, install flow, doctor reading guide, performance expectations table, links to share-rate calculators
- `docs/development/mining-providers.md` — generalized "how to add a new provider" guide using this spec as the canonical worked example
- An update to `docs/user-guide/mining.md` (Spec A) adding Apple Silicon to the supported-platforms list
### 8.4 Exit criteria
- [ ] Hardware matrix covered
- [ ] Testnet bringup complete
- [ ] Documentation merged
- [ ] Pearl maintainer sign-off obtained
- [ ] OJ release notes call out Apple Silicon mining as supported
## 9. Risks
| ID | Risk | Likelihood | Impact | Mitigation |
|---|---|---|---|---|
| R1 | Pearl validator rejects non-CUDA-mined blocks despite hardware-neutral validator code | low (validator code reviewed) | catastrophic (whole spec invalid) | Phase 0-A explicit confirmation; Phase 0-B oracle reduces likelihood of math drift |
| R2 | Pearl ships their own Metal port, conflicts with OJ's | medium (depends on Pearl roadmap) | high (rework or fork) | Phase 0-A coordination; default to upstream contribution |
| R3 | Metal NoisyGEMM is so slow that mining is uneconomical even for hobbyists | medium-high | medium (feature ships but unused) | §4.3 names this as a non-goal; transparency in `mine doctor`; consider M-Ultra-only-by-default in v1 of this spec |
| R4 | NoisyGEMM correctness bug → invalid blocks → wasted user electricity | low if §6.1.4 testing rigorous | high (trust hit) | Strong parity testing against oracle; testnet bringup before mainnet |
| R5 | Inference-quality regression — denoised path doesn't fully recover model fidelity | medium | high | §6.2.3 regression tests; eval-harness gate before ship |
| R6 | Pearl protocol changes between Phase 0 and Phase 4 (multi-month) | medium | medium | Pin Phase 0 ref same as Spec A; renegotiate at each Pearl rev |
| R7 | Apple changes Metal API in macOS update | low-medium | medium | Use stable MSL features; pin Xcode toolchain |
| R8 | Upstream Pearl PR rejected | low (Pearl wants this) | medium (forced fork) | Phase 0-A negotiates upstream-vs-fork up front |
| R9 | `pearl-gateway` doesn't build on Apple Silicon (§7.5) | medium (it's Python — should work, but py-pearl-mining has Rust deps) | low (small fix) | Phase 0 verifies builds; Phase 0-A coordination if not |
## 10. Open questions
Phase 0 answered most of these from the upstream code (annotations below). The remaining open items are ones that require either Pearl maintainer input or empirical measurement on real network conditions.
1. **(Open — coordination)** Is there a Pearl-blessed "small" model for testing? The reference miner uses a 70B model — too big for fast iteration. A 7B or 13B variant for development would dramatically speed up v2/v3 plugin work. *Less critical for v1, since v1 is decoupled from inference and runs `mine()` directly without a model.*
2. **(Answered — N/A for v1)** ~~Documented fp16 tolerance budget for denoised matmul output~~ — `miner-base/tests/test_noisy_gemm.py:92` does `torch.equal(result, expected)`: the int7×int7→int32 path is **bit-exact**, no fp16 tolerance budget is needed. (May reappear in v3 Metal kernel work if int↔fp16 conversions are introduced for perf.)
3. **(Answered — yes)** Does `py-pearl-mining` already expose enough of NoisyGEMM in Python that Phase 0-B becomes a thin wrapper? **Yes.** `pearl_mining.mine` runs the entire mining algorithm in pure Rust. Additionally, `miner-base.NoisyGemm` provides a PyTorch reference of the production NoisyGEMM. Phase 0-B's "build a reference oracle" deliverable is now a *thin OJ-side wrapper* that calls upstream — see §13 and `tools/pearl-reference-oracle/` (created in this session).
4. **(Answered — likely yes; empirically verified for `py-pearl-mining`)** Is `pearl-gateway` cross-platform? Its `pyproject.toml` requires Python ≥ 3.10 and depends on `aiohttp`, `bitcoin-utils`, `blake3`, `numpy`, `prometheus-client`, `pybase64`, `pydantic`, `pyyaml`, `torch==2.11.0`, `py-pearl-mining` — all install on macOS arm64. Empirical install of the workspace was not run in this session; **action item for v1 implementation**: `uv sync` the workspace on macOS-15 and capture the build output.
5. **(Open — coordination)** Does Pearl gate any difficulty / consensus parameters on hardware introspection? Code review found none. Confirm in Phase 0-A discussion.
6. **(Open — measurement)** Minimum acceptable hashrate floor for `jarvis mine init` on Apple Silicon. The `MiningCapabilities.estimated_hashrate` field in Spec A §4.4 exists for this. v1 will populate from a calibration run during `mine init`. The *floor* is a policy decision, not a technical one — defer to user-research / community feedback once v1 ships.
7. **(Open — coordination)** Upstream contribution / CLA / LICENSE. Pearl is ISC; OJ is Apache-2.0; both are permissive and combine cleanly. **CLA TBD via Phase 0-A discussion**, but for v1 this is moot — OJ contributes no code into Pearl's tree, only consumes their published Python packages.
8. **(Answered — yes)** Apple Silicon CI on GitHub Actions: `macos-14` / `macos-15` runners are arm64 and can install `py-pearl-mining` via the wheel build verified in §1.5.4. OJ's CI can run mining unit tests. Mining the *real* network in CI is still out of scope.
9. **(Answered — `"llamacpp"`)** OJ's llama.cpp engine_id is `"llamacpp"` (single token, no hyphen). Confirmed at `src/openjarvis/engine/openai_compat_engines.py:9` and `src/openjarvis/engine/_discovery.py:18`. Update §7.3 capability detection to use this key. *(For v1 in §13, this only matters if we add an "informational" mining-aware hint to the existing llamacpp engine — v1 does not require any plugin into the engine.)*
10. **(Open — measurement, but de-risked)** plonky2 STARK proving latency on Apple Silicon CPU. Spec A §1 already notes proving is seconds-to-minutes of CPU per block (cross-platform, runs unchanged). For v1 the hashrate is so low that block-find latency is dominated by the search, not the proof. Empirical measurement still needed for v2/v3.
11. **(New — v1 specific)** Does `bitcoin-utils>=0.7.0` (a `pearl-gateway` dependency) have C extensions that need Apple-specific build flags? Likely pure-Python; verify during the v1 install workstream.
12. **(New — v1 specific)** Will `torch==2.11.0` (the version `miner-base` and `pearl-gateway` pin) install cleanly on macOS arm64? PyTorch generally has arm64 macOS wheels. Verify during v1 install.
## 11. Cross-references
- **[Spec A](2026-05-05-vllm-pearl-mining-integration-design.md)** — the v1 integration this extends. Read §4.4 (the `MiningProvider` ABC), §5.3 (sidecar shape), §8.1–8.2 (telemetry adapter contract), §8.5 (v2 fee/pool seams). All apply unchanged.
- **Pearl coordination thread (P0-A draft):** [`2026-05-05-pearl-coordination-discussion-draft.md`](2026-05-05-pearl-coordination-discussion-draft.md) — content the user posts on `pearl-research-labs/pearl` to confirm protocol acceptance and align on contribution model.
- **OJ-side Phase 0 deliverables (created this session):**
- `tools/pearl-reference-oracle/` — thin Python wrapper around upstream Pearl bindings + smoke test, runnable on Apple Silicon
- **Pearl repo paths read in Phase 0 (in priority order):**
1. `pearl/zk-pow/src/api/verify.rs` — the validator. Pure Rust, no GPU. **Hardware-neutrality verified.**
2. `pearl/zk-pow/src/api/proof.rs` — `PublicProofParams`, `ZKProof`, `PrivateProofParams`, `IncompleteBlockHeader`, `MiningConfiguration`, `MMAType`. Defines what the protocol commits to.
3. `pearl/zk-pow/src/ffi/mine.rs` — **the entire hardware-neutral mining function**. Pure Rust. Already exposed to Python.
4. `pearl/zk-pow/src/circuit/pearl_noise.rs` — noise generation: `compute_noise_for_indices`, `generate_uniform_random_matrix`, `generate_permutation_matrix`. Hardware-neutral.
5. `pearl/py-pearl-mining/src/lib.rs` — PyO3 module. Re-exports `mine`, `verify_plain_proof`, `generate_proof`, `verify_proof`, `warmup_prove`. **Builds on macOS arm64, verified §1.5.4.**
6. `pearl/py-pearl-mining/Cargo.toml` — pure Rust deps: `pearl-blake3`, `zk-pow`, `blake3`, `rayon`, `pyo3`, `lazy_static`, `tikv-jemallocator`. No CUDA in tree.
7. `pearl/py-pearl-mining/tests/test_python_api.py` — the canonical end-to-end test. Use as the OJ smoke-test template.
8. `pearl/miner/miner-base/src/miner_base/noisy_gemm.py` — PyTorch reference of the production NoisyGEMM. The "reference oracle" §5.2 wanted to build is here.
9. `pearl/miner/miner-base/src/miner_base/noise_generation.py` — PyTorch noise generation matching `pearl_noise.rs`.
10. `pearl/miner/miner-base/src/miner_base/inner_hash.py` — PyTorch inner-hash with XOR reduction.
11. `pearl/miner/miner-base/tests/test_noisy_gemm.py` — bit-exact denoising verified at line 92.
12. `pearl/miner/miner-base/pyproject.toml` — deps (`torch==2.11.0`, `blake3`, `numpy`, `pearl-gateway`, `py-pearl-mining`); **no platform markers** → installs on Apple Silicon.
13. `pearl/miner/pearl-gateway/pyproject.toml` — deps (pure Python + py-pearl-mining + torch); **no platform markers**.
14. `pearl/miner/vllm-miner/src/vllm_miner/register.py` — vLLM plugin registration via `vllm.general_plugins` entry point. The pattern Phase 2 (v2 plan) would mirror.
15. `pearl/miner/pearl-gemm/csrc/gemm/pearl_gemm_constants.hpp` — protocol scale factors. Verified values:
- `kAxEBLScaleFactor = 1<<14 = 16384`
- `kEARxBpEBScaleFactor = 1<<12 = 4096`
- `kIntToFp16ScaleFactor = 1<<12 = 4096`
- `kEBRScaleFactorDenoise = -4` (= -kAxEBLScaleFactor / kIntToFp16ScaleFactor)
- `kEALScaleFactorDenoise = -1` (= -kEARxBpEBScaleFactor / kIntToFp16ScaleFactor)
16. `pearl/miner/pearl-gemm/setup.py:88` — `COMPUTE_CAPABILITY = "arch=compute_90a,code=sm_90a"`. Confirms CUDA kernel is Hopper-only.
17. `pearl/Taskfile.yml` — `build:miner` task is gated to `platforms: [linux, windows]`. **The miner Python install path Pearl ships today is Linux/Windows-only**; OJ's v1 path uses the components that *do* install on macOS, sidestepping this gate.
- **Pearl paper:** [Proof-of-Useful-Work via matrix multiplication (arXiv:2504.09971)](https://arxiv.org/abs/2504.09971) — read for the math formalization. Less critical now that the PyTorch reference exists upstream.
- **Apple references (still relevant for v2/v3):**
- [Metal Shading Language Specification](https://developer.apple.com/metal/Metal-Shading-Language-Specification.pdf)
- [MPS / MPSGraph documentation](https://developer.apple.com/documentation/metalperformanceshadersgraph)
- [MLX](https://github.com/ml-explore/mlx) — alternative v2 plugin host
- [PyTorch MPS backend docs](https://pytorch.org/docs/stable/notes/mps.html) — relevant for v2 (MPS-accelerated decoupled mining)
## 12. Implementation plan
Two implementation plans now live alongside this spec:
- **v1 plan (decoupled CPU mining via upstream Pearl):** Written via `superpowers:writing-plans` after this Phase 0 update. Tracking issue: [`2026-05-05-apple-silicon-pearl-mining-plan-v1.md`](2026-05-05-apple-silicon-pearl-mining-plan-v1.md).
- **v2 plan (PyTorch-MPS or MLX/llama.cpp coupled mining):** TBD. Written when v1 ships and we have empirical hashrate data justifying the next investment.
- **v3 plan (native Metal NoisyGEMM kernel):** TBD. Written only if v2 measurements show the additional kernel work is economically justified.
The original §5–§8 content describing Phases 0–4 of the *kernel-first* approach is preserved as the v3 plan's reference. Do not delete it — when the time comes to write the v3 plan, that content is the starting point.
## 13. Apple Silicon v1 — minimal path
This section defines the v1 design that ships in weeks rather than months. v1 is **decoupled mining**: the user's existing inference workflow (Ollama, MLX-LM, llama.cpp, vLLM-on-CPU, anything) is untouched; mining runs as a separate process via the upstream Pearl miner.
### 13.1 Architecture
```
OpenJarvis user (Apple Silicon)
┌────────────────────────────────┐
│ jarvis mine start │
│ ↓ │
│ CpuPearlProvider (this spec) │
│ ↓ subprocess.Popen │
│ ┌──────────────────────────┐ │
│ │ pearl-gateway (Python) │ │
│ │ ↑ JSON-RPC :8337 │ │
│ │ pearl-mine-loop (Python) │ │ ← uses py-pearl-mining
│ │ wraps pearl_mining.mine│ │ (pure Rust)
│ └──────────────────────────┘ │
│ │
│ Inference (untouched) │
│ ┌──────────────────────────┐ │
│ │ Ollama / MLX / llamacpp │ │
│ └──────────────────────────┘ │
└────────────────────────────────┘
↓
pearld (BYO, same as Spec A)
```
The mining loop wraps `pearl_mining.mine()` in a process that:
1. Polls `pearl-gateway` for current `IncompleteBlockHeader` + `MiningConfiguration`
2. Calls `pearl_mining.mine(m, n, k, header, config)` to find a `PlainProof`
3. Submits the proof back to `pearl-gateway`, which generates the ZK proof and forwards to `pearld`
4. Loops
This is the *same* control flow vllm-miner runs — just without coupling the matmul to vLLM's inference. Pearl's existing `pearl-gateway` already does the orchestration we need; we just need a small mining loop that uses the CPU `mine()` instead of the CUDA path.
### 13.2 Module layout in OJ
```
src/openjarvis/mining/
cpu_pearl.py # @MinerRegistry.register("cpu-pearl") — v1 provider
_pearl_subprocess.py # PearlSubprocessLauncher — gateway + miner subprocesses
# Reused for future Apple-MPS / Metal providers
src/openjarvis/cli/
# mine_cmd.py is unchanged; cpu-pearl participates via the provider ABC
tests/mining/test_cpu_pearl.py
tools/pearl-reference-oracle/
README.md # documentation: oracle exists upstream
smoke_test.py # end-to-end mine + verify smoke test (created in this session)
```
### 13.3 Optional extra
```toml
mining-pearl-cpu = [
"py-pearl-mining>=0.1", # the wheel built in §1.5.4
"miner-base>=0.1", # PyTorch reference (used for parity testing)
"pearl-gateway>=0.1", # gateway service
]
```
When Pearl publishes these as PyPI wheels, the install is `uv sync --extra mining-pearl-cpu`. Until then, the spec for the implementation plan covers the local-build fallback (clone Pearl at the pinned ref, `maturin build` `py-pearl-mining`, `uv pip install` the workspace packages from local paths).
### 13.4 Capability detection
```python
# src/openjarvis/mining/cpu_pearl.py
class CpuPearlProvider(MiningProvider):
provider_id = "cpu-pearl"
@classmethod
def detect(cls, hw: HardwareInfo, engine_id: str, model: str) -> MiningCapabilities:
# cpu-pearl is engine-independent — it doesn't plug into inference
if not _pearl_mining_available():
return MiningCapabilities(False, reason="install with `uv sync --extra mining-pearl-cpu`")
if not _pearl_gateway_available():
return MiningCapabilities(False, reason="pearl-gateway package not installed")
if hw.platform not in {"darwin", "linux"}:
return MiningCapabilities(False, reason=f"platform '{hw.platform}' not yet supported")
# Optional: hardware-specific hashrate estimates
return MiningCapabilities(True, estimated_hashrate=_estimate_cpu_hashrate(hw))
```
The `engine_id` parameter is ignored because v1 is decoupled — mining works with **any** OJ engine, including no engine at all. (A future Apple-coupled provider would inspect `engine_id` to require `"llamacpp"` or `"mlx"`.)
### 13.5 Lifecycle (from Spec A's `MiningProvider` ABC)
- `start(config)`: spawn (1) `pearl-gateway` and (2) `pearl-mine-loop` subprocesses. Wait for gateway readiness on `:8339/metrics`. Write the standard sidecar JSON (Spec A §5.3) with `provider="cpu-pearl"`, gateway URL, and PIDs of both subprocesses.
- `stop()`: SIGTERM mining loop, then gateway. Bounded waits, SIGKILL fallback.
- `is_running()`: check sidecar + both PIDs.
- `stats()`: read from `pearl-gateway`'s `:8339/metrics` exactly as Spec A §8.1 specifies. **Same metrics adapter contract.** No code changes in OJ's gateway-metrics adapter.
### 13.6 Configuration
Inherits Spec A's `[mining]` config schema unchanged. v1 uses:
```toml
[mining]
provider = "cpu-pearl" # NEW: was "vllm-pearl" in Spec A
wallet_address = "prl1q..."
submit_target = "solo"
fee_bps = 0
fee_payout_address = ""
[mining.extra]
gateway_port = 8337
metrics_port = 8339
pearld_rpc_url = "http://localhost:44107"
pearld_rpc_user = "rpcuser"
pearld_rpc_password_env = "PEARLD_RPC_PASSWORD"
# v1-specific: matmul shape for the search loop
m = 256
n = 128
k = 1024
rank = 32
```
The `m / n / k / rank` shape can be tuned per Phase 0-A measurement (or per chip). Larger shapes search more space per call but use more memory.
### 13.7 Doctor surface (Apple Silicon)
```
$ jarvis mine doctor
Hardware
GPU vendor apple ✓
Apple chip M2 Max ✓
Unified memory 96 GB ✓
Pearl install
py-pearl-mining 0.1.0 (cp312-abi3-macos-arm64) ✓
miner-base 0.1.0 ✓
pearl-gateway 0.1.0 ✓
Pearl node
RPC http://localhost:44107 ✓
Auth ok ✓
Block height 442107 (synced) ✓
Wallet
Address format prl1q... ✓
Provider capability
cpu-pearl SUPPORTED (est. 0.X share/h on M2 Max)
Notes
- This is decoupled mining: your normal LLM inference is unaffected
- Hashrate is far below H100 mining; see docs/user-guide/mining-apple-silicon.md
- Metal-accelerated mining: planned for v2; not available yet
Session
Sidecar absent (not running)
```
Each row maps to a check function in `mining/_discovery.py`. The "est. share/h" line is populated from a one-time calibration during `mine init` — runs `pearl_mining.mine` in a 30-second loop and extrapolates.
### 13.8 v1 anti-goals
- **No coupling to inference.** The user's MLX-LM / Ollama / llama.cpp inference is untouched. v1 does not introduce a custom matmul. The "use AI = mine" narrative is **explicitly deferred to v2**.
- **No Metal kernel.** All math is in upstream Rust + PyTorch + Python. Zero MSL written.
- **No Pearl tree changes.** We consume their published packages; we contribute zero code into Pearl's repo for v1. (Phase 0-A discussion still happens — but it's lower-stakes since we're a downstream consumer in v1, not a contributor.)
- **No upstream PRs blocking v1 ship.** v1 ships against the Pearl ref pinned in `mining/_constants.py` (Spec A §6) regardless of whether any of our coordination questions are answered.
### 13.9 v1 exit criteria
- [ ] `mining-pearl-cpu` extra installs cleanly on macOS arm64 (M1, M2, M3, M4 — at minimum the chip the spec author owns)
- [ ] `jarvis mine init` completes successfully on macOS arm64
- [ ] `jarvis mine start` launches gateway + miner subprocesses; sidecar valid; `mine status` reports live data
- [ ] `mine doctor` produces honest, actionable output for Mac users
- [ ] At least one block found on Pearl testnet from at least one Apple Silicon variant
- [ ] User-facing doc `docs/user-guide/mining-apple-silicon.md` ships, including the honest hashrate caveat
### 13.10 Out of v1, into v2/v3
- **v2 (months):** Re-route `noisy_gemm` math to PyTorch-MPS for Apple Silicon GPU acceleration; integrate as a plugin into MLX-LM or `llama-cpp-python` so inference matmuls produce mining work (preserving the "use AI = mine" narrative). The original §5–§8 plan applies, swapped to use PyTorch MPS instead of raw MSL.
- **v3 (months — optional, only if v2 perf is insufficient):** Native Metal Shading Language NoisyGEMM kernel as an upstream Pearl contribution. The original §5–§8 plan applies as written.
## 14. Phase 0 deliverables status (this session, 2026-05-05)
Tracking what was actually produced, against the §5 Phase 0 plan and the §1.5 reframing.
| Workstream | Original plan | Status | Deliverable |
|---|---|---|---|
| P0-A | Open Pearl GitHub Discussion, get protocol-acceptance confirmation | Draft written; user posts | `docs/design/2026-05-05-pearl-coordination-discussion-draft.md` |
| P0-B | Build reference oracle from scratch in PyTorch, validate against H100 CUDA | **Reference oracle exists upstream.** Built thin OJ-side wrapper + verified empirically that `pearl_mining.mine` runs on Apple Silicon (78 ms / proof at test difficulty) | `tools/pearl-reference-oracle/` |
| P0-C | Decide MLX vs llama.cpp Metal | **Deferred to v2.** v1 doesn't need either. | — |
| Spec update | Capture findings | Done | This document, §1.5, §10–§14 |
| v1 implementation plan | Plan written via `superpowers:writing-plans` after Phase 0 | Pending | `2026-05-05-apple-silicon-pearl-mining-plan-v1.md` (next deliverable) |
docs/design/2026-05-05-apple-silicon-pearl-mining-plan-v1.md
+1811
View File
File diff suppressed because it is too large Load Diff
docs/design/2026-05-05-pearl-coordination-discussion-draft.md
+91
View File
@@ -0,0 +1,91 @@
# Pearl coordination thread — draft
**For:** Posting on `pearl-research-labs/pearl` GitHub Discussions (Category: General / Q&A).
**By:** OpenJarvis team (Stanford Hazy Research); contact: [user fills in].
**Status:** Draft — review and edit before posting.
---
## Suggested title
> Apple Silicon support for Pearl mining — coordination & confirmation
## Suggested body
Hi Pearl team — we're [OpenJarvis](https://github.com/open-jarvis/OpenJarvis), a local-first personal AI agent framework from Stanford Hazy Research. We're working on a `mining` subsystem that lets OJ users mine Pearl through the agent framework. The first integration is the `vllm-miner`-on-H100/H200 path, which is straightforward. The second is Apple Silicon, where the situation is more interesting and we'd like to confirm a few things before we ship.
We have a v1 architecture that ships **today** using only your published Python packages (`py-pearl-mining`, `miner-base`, `pearl-gateway`) without any new code in your tree, plus an aspirational v2/v3 path that does involve potentially upstream contributions. Three asks below, plus a heads-up.
### What we built and verified locally (no protocol changes; all upstream code paths)
We read the Pearl source carefully — particularly:
- `zk-pow/src/api/verify.rs` — the validator
- `zk-pow/src/ffi/mine.rs` — the pure-Rust `mine()` function
- `zk-pow/src/circuit/pearl_noise.rs` — noise generation
- `py-pearl-mining/` — the PyO3 bindings exposing the above to Python
- `miner/miner-base/src/miner_base/noisy_gemm.py` — the PyTorch NoisyGEMM reference
…and then we built `py-pearl-mining` from source on an Apple Silicon M2 Max (macOS 26.4, Python 3.12, Rust 1.94). It produced `py_pearl_mining-0.1.0-cp312-abi3-macosx_11_0_arm64.whl` in ~56 seconds. We installed it and ran the `mine()` + `verify_plain_proof()` cycle from `tests/test_python_api.py`:
```
running mine(m=256, n=128, k=1024, rank=32) on Apple Silicon CPU…
mine() returned a proof in 0.078s
verify_plain_proof: ok=True, msg='Mining solution verified successfully'
```
So our v1 plan is: ship a CPU-mining mode for OJ users on Apple Silicon (and potentially other non-CUDA platforms) that wraps `pearl_mining.mine()` and your `pearl-gateway` as a subprocess. **We're not modifying anything in Pearl's tree for v1.** Just consuming what you've already published.
### Three asks
**1. Protocol acceptance confirmation.**
Reading the validator path, we believe `verify_block` and `verify_plain_proof` accept any `PlainProof` produced by a correct implementation, regardless of which hardware produced it. The plonky2 STARK and the difficulty check don't reference hardware.
**Could you confirm in writing that blocks mined via the pure-Rust `mine()` path (from a non-CUDA host like Apple Silicon) will be accepted by Pearl validators on testnet and mainnet?** We don't expect surprises here, but it's load-bearing for our spec and we want to record your sign-off before we ship.
**2. Heads-up: your `Taskfile.yml` restricts `build:miner` to `[linux, windows]`.**
That makes total sense for the GPU miner (CUDA + vLLM is Linux-only). But the `py-pearl-mining` and `miner-base` packages don't actually need that restriction — they install fine on macOS. We're working around the gate by installing the individual packages directly. Two questions:
- Is the `[linux, windows]` restriction load-bearing in some way we don't see (e.g., do you intend `py-pearl-mining` to remain a CUDA-bound dependency long-term)?
- Would you be open to a small PR that splits `build:miner-cpu` (cross-platform) from `build:miner-gpu` (Linux + CUDA)? It would help downstream consumers like us — and any hobbyist who wants to experiment with `pearl_mining.mine()` on whatever hardware they own.
**3. PyPI publication of `py-pearl-mining` / `miner-base` / `pearl-gateway`.**
Do you have a roadmap for publishing these as PyPI wheels (`pip install py-pearl-mining` etc.)? Today we'd vendor a pinned commit and `maturin build` locally, which works but is brittle. If a 2026 PyPI publication is plausible, we'd defer the local-build code path; if it's not on the roadmap, we'll plan for the long-term local-build path.
### Aspirational (v2 / v3) — context only, no asks yet
Once v1 ships, we'd like to explore Apple-native acceleration:
- **v2:** Use PyTorch MPS to GPU-accelerate `miner-base.NoisyGemm` on Apple Silicon. Could potentially become a plugin into `mlx-lm` or `llama-cpp-python` so a Mac user's *inference* matmuls do mining work — same "useful work" framing as your vllm-miner. We don't need anything from Pearl for this; we'd build it on top of your existing PyTorch reference.
- **v3 (only if v2 isn't enough):** A native Metal Shading Language port of NoisyGEMM, paralleling `pearl-gemm/`. That would be a real upstream contribution candidate (`pearl/miner/pearl-gemm-metal/`), and we'd want to coordinate with you before starting kernel work to avoid duplicate effort.
If you're already building Apple Silicon support internally (or have someone planning it), please tell us — we'd rather coordinate than duplicate.
### Logistics
- License compatibility: Pearl is ISC; OpenJarvis is Apache-2.0. We don't see any conflict for either consumption (v1) or contribution (v3), but please flag if you do.
- CLA: do you require one for upstream contributions? Not blocking v1 — just want to know for v3.
- Preferred coordination channel: this Discussion thread, a Discord, an email? We're happy to use whatever works for you.
Thanks for building this — Proof-of-Useful-Work via matmul is genuinely interesting and we're excited to bring more (slower!) hardware to the network.
— [user name], on behalf of OpenJarvis
---
## Notes for the user before posting
- Replace `[user fills in]` with your contact info, `[user name]` with your name.
- The architecture/perf claims are all backed by code + an actual local build; you can stand behind them.
- "Heads-up" framing on the `Taskfile.yml` is intentional — we're not asking them to *change* it, just flagging the friction point in case they want to.
- Don't post until OJ Spec A is at least branch-pushed (which it is, PR #310) — it gives Pearl a way to see the broader integration we're building.
- When their reply lands, update Spec B §10 (open questions 1, 5, 7) and §11 (cross-references → coordination thread URL).
## Possible Pearl responses to anticipate
- **Best case:** "Confirmed, looks great, we don't have an Apple Silicon plan, please do it." — proceed with §13.
- **Middle case:** "Confirmed, but we have a Metal port in flight." — coordinate, share Spec B §6.1, decide upstream-vs-fork. v1 (CPU) is unaffected.
- **Worst case:** "We'd prefer downstream non-CUDA mining stay disabled for now." — unlikely given their `pearl-gateway` README explicitly anticipates "plugins for other LLM inference libraries", but if it happens, this becomes a much harder problem and we'd need to revisit.
docs/design/2026-05-05-vllm-pearl-mining-integration-design.md
+560
View File
@@ -0,0 +1,560 @@
# Spec A — vLLM-Pearl mining integration (v1)
| | |
|---|---|
| **Date** | 2026-05-05 |
| **Status** | Design — pending implementation plan |
| **Owner** | OpenJarvis team |
| **Companion spec** | [Spec B — Apple Silicon enablement](2026-05-05-apple-silicon-pearl-mining-design.md) (separate effort, runs in parallel) |
| **Repos referenced** | `OpenJarvis` (this repo), `pearl-research-labs/pearl` |
## 1. Summary
Add a new sibling subsystem `openjarvis.mining` that lets users run [Pearl](https://github.com/pearl-research-labs/pearl) Proof-of-Useful-Work mining as a property of their local LLM inference. v1 ships solo mining for users who already have an H100/H200 and run vLLM — the only configuration Pearl's reference miner currently supports. The architecture leaves three deliberate seams for v2 (pool support + a 20% OJ fee) and is engine-agnostic by construction so Apple Silicon, AMD, Ollama, llama.cpp, and MLX paths plug in via the registry without a rewrite when Pearl ships the matching plugins.
The narrative thesis: Pearl's `vllm-miner` is a vLLM plugin that swaps quantized linear ops with `NoisyGEMM`, a CUDA kernel that produces both the correct matmul output *and* a PoW commitment. Mining IS inference. For an OJ user already serving prompts on a powerful local GPU, this is a way to capture economic value from compute they were going to do anyway — directly aligned with OJ's Intelligence-Per-Watt thesis rather than against it.
## 2. Scope
### In scope (v1)
- New `openjarvis.mining` subsystem with `MiningProvider` ABC, `MinerRegistry`, `MiningCapabilities` / `MiningConfig` / `MiningStats` dataclasses
- `vllm-pearl` provider implementation: orchestrate Pearl's published `vllm-miner` Docker container
- `[mining]` TOML section in OJ config; `MiningConfig` field in `JarvisConfig`
- New CLI namespace: `jarvis mine init|start|stop|status|doctor|attach|logs`
- Runtime sidecar at `~/.openjarvis/runtime/mining.json` for engine ↔ mining handoff
- Hybrid Docker image acquisition: pull-if-published, otherwise build from a pinned Pearl ref
- On-demand telemetry via Pearl gateway `:8339/metrics`; `mining_session_id` nullable column on telemetry inference rows
- v2 seams: `submit_target` tagged-union parsing, zero-valued `fee_bps` / `fees_owed` plumbing, reserved `mining/pools/` location
- Test strategy that doesn't require an H100 in CI
- Documentation: `docs/user-guide/mining.md`, `docs/development/mining.md`, `CLAUDE.md` paragraph, `REVIEW.md` bullet
### Out of scope (v1) — deferred or owned elsewhere
- Pool support and the 20% OJ fee mechanism (own spec, v2)
- Custody, signing, or routing Pearl funds (anti-goal — must remain zero in v1)
- Apple Silicon, AMD ROCm, sm89 (RTX 4090) NVIDIA, CPU, MLX, Ollama, llama.cpp, SGLang mining paths (Spec B for Apple; remaining hardware/engine paths blocked on Pearl)
- Wallet generation, Oyster integration, key custody (paste-only address)
- pearld lifecycle management (BYO node)
- Background telemetry collection in OJ's gateway daemon (v1.x; the hook point is reserved)
- Inference-quality drift detection (v1.x at earliest)
- `mine doctor --fix` automatic remediation (v1.x stub)
- Multi-GPU / multi-worker / multi-session per host (v2+)
## 3. Load-bearing decisions from brainstorming
These were the forks where the design could have gone several ways. Recorded so future-readers can audit reasoning rather than re-derive it.
| Decision | What we picked | Why |
|---|---|---|
| Target audience for v1 | H100/H200 owners running vLLM (Pearl's only working config today) | Anything broader is blocked on Pearl shipping non-CUDA / non-vLLM plugins. Power-user MVP ships in weeks; pool/fee/Apple are separate specs. |
| Mining model | Co-located: every inference through the Pearl-flavored vLLM is mining work | Matches Pearl's `vllm-miner` plugin design and OJ's Intelligence-Per-Watt thesis. Side-car deferred until Pearl ships plugins for engines users care about for non-mining inference. |
| Coupling to Pearl miner process | Wrap-and-launch via Docker | Pearl's Docker image (or Dockerfile) is the most stable contract they expose. (1) "BYO miner" is too thin to be a feature; (3) running Pearl's `uv` workspace natively couples us to their build system. |
| Module placement | Sibling top-level subsystem `mining/` (peer to `engine/`, `agents/`) | Matches OJ's existing module pattern. `MinerRegistry` is a peer registry. Future non-vLLM providers slot in identically. |
| Engine attachment | Runtime sidecar JSON at `~/.openjarvis/runtime/mining.json` | Existing vLLM engine class stays untouched. Sidecar is the single source of truth tying mining lifecycle to engine resolution. Inspectable via `cat`. |
| Config shape | Flat top-level `[mining]` TOML section | Only one provider in v1; nested per-engine config can grow later if multi-provider becomes real. |
| Wallet handling | Paste-only Pearl Taproot address | Keys are sensitive; Pearl's wallet RPC is unstable surface. v1.x can add Oyster integration once the contract stabilizes. |
| pearld | BYO; user points OJ at their own node | OJ doesn't orchestrate L1 nodes. Doctor surfaces unreachable cleanly. |
| Telemetry collection | On-demand reads in v1; persistent collector class shipped unwired (`MiningTelemetryCollector`) | Most users won't enable mining; daemon shouldn't grow surface for them. v1.x lights up the hook with zero API churn. |
| v1 fee/pool seams | Three seams: `submit_target` parsed (one variant works), `fee_bps`/`fees_owed` plumbed at zero, `mining/pools/` reserved | Cheap to leave; painful to retrofit. Does not pre-decide the v2 API. |
| Custody | **Anti-goal**: zero. v1 must not accept, sign, or route Pearl funds. | Avoids prematurely binding a legal/regulatory posture. v2 revisits as part of pool design. |
| Apple Silicon support | Not in v1. Designed-for via the `MiningProvider` ABC + `MiningCapabilities.detect()`. Spec B documents the enablement work. | The Pearl `pearl-gemm` kernel is heavily Hopper-bound (`sm_90a`, WGMMA, TMA, cluster mode, CUTLASS 3.x). A Metal port is real GPU-kernel engineering, not a config flag. |
## 4. Architecture & module layout
### 4.1 New module tree
```
src/openjarvis/mining/
__init__.py # soft-imports providers (try/except ImportError)
_stubs.py # MiningProvider ABC + dataclasses (MiningCapabilities, MiningConfig, MiningStats, SoloTarget, PoolTarget)
_discovery.py # detect_providers(hardware, engine, model) -> list[MiningCapabilities]
_docker.py # PearlDockerLauncher — shared Docker orchestration (image acquisition + container lifecycle)
_collector.py # MiningTelemetryCollector class — defined but UNWIRED in v1; lit up in v1.x
_constants.py # PEARL_REPO, PEARL_PINNED_REF, PEARL_IMAGE_TAG, OJ default tag
vllm_pearl.py # @MinerRegistry.register("vllm-pearl") — only impl in v1
pools/ # RESERVED for v2. Empty in v1 except for an __init__.py with a docstring saying so.
src/openjarvis/cli/
mine_cmd.py # jarvis mine init|start|stop|status|doctor|attach|logs
tests/mining/
__init__.py
conftest.py # mining-specific fixtures (synthetic HardwareInfo, sample Prometheus output)
fixtures/
gateway_metrics_sample.txt # captured Prometheus output from a real Pearl run
config_*.toml # golden TOML files
test_stubs.py
test_discovery.py
test_docker.py
test_collector.py
test_vllm_pearl.py
test_cli.py
```
### 4.2 Registry additions
`MinerRegistry` added to `src/openjarvis/core/registry.py` as a peer to `EngineRegistry`, `AgentRegistry`, etc. `tests/conftest.py`'s autouse `_clean_registries` fixture is updated to include `MinerRegistry.clear()`.
`mining/vllm_pearl.py` exposes idempotent `ensure_registered()`:
```python
def ensure_registered() -> None:
if not MinerRegistry.contains("vllm-pearl"):
MinerRegistry.register_value("vllm-pearl", VllmPearlProvider)
```
`mining/__init__.py` soft-imports `vllm_pearl` inside `try / except ImportError` and calls `ensure_registered()`. Standard OJ pattern.
### 4.3 Optional-deps extras
```toml
mining-pearl = ["docker>=7.0"] # v1 requires only the Docker SDK
# mining-pearl-mlx = [...] # future, owned by Spec B
# mining-pearl-rocm = [...] # future
```
### 4.4 The central ABC
```python
# src/openjarvis/mining/_stubs.py
from abc import ABC, abstractmethod
from dataclasses import dataclass, field
from openjarvis.core.config import HardwareInfo
@dataclass(slots=True)
class MiningCapabilities:
supported: bool
reason: str | None = None # human-readable: "needs sm90", "no Pearl plugin for engine ollama"
estimated_hashrate: float | None = None
@dataclass(slots=True)
class SoloTarget:
pearld_rpc_url: str
@dataclass(slots=True)
class PoolTarget:
url: str
worker_id: str | None = None
SubmitTarget = SoloTarget | PoolTarget
@dataclass(slots=True)
class MiningConfig:
provider: str # MinerRegistry key
wallet_address: str
submit_target: SubmitTarget # parsed from TOML "solo" / "pool:<url>"; v1 accepts only SoloTarget at runtime
fee_bps: int = 0 # v1: 0; v2: 2000 (=20%)
fee_payout_address: str | None = None # v1: ignored; v2: OJ's address
extra: dict = field(default_factory=dict)
@dataclass(slots=True)
class MiningStats:
provider_id: str
shares_submitted: int = 0
shares_accepted: int = 0
blocks_found: int = 0
hashrate: float = 0.0
uptime_seconds: float = 0.0
last_share_at: float | None = None
last_error: str | None = None
payout_target: str = "solo" # v2 reporting; "solo" in v1
fees_owed: int = 0 # v2 accounting hook; 0 in v1
class MiningProvider(ABC):
provider_id: str
@classmethod
@abstractmethod
def detect(cls, hw: HardwareInfo, engine_id: str, model: str) -> MiningCapabilities: ...
@abstractmethod
async def start(self, config: MiningConfig) -> None: ...
@abstractmethod
async def stop(self) -> None: ...
@abstractmethod
def is_running(self) -> bool: ...
@abstractmethod
def stats(self) -> MiningStats: ...
```
## 5. Config schema & engine attachment
### 5.1 TOML schema
```toml
[mining]
provider = "vllm-pearl" # MinerRegistry key
wallet_address = "prl1q..." # user's Pearl Taproot address (paste-only)
submit_target = "solo" # v1: "solo" only; "pool:<url>" raises NotImplementedError at start()
fee_bps = 0 # v1: 0; v2: 2000
fee_payout_address = "" # v1: ignored; v2: OJ's address
[mining.extra]
docker_image_tag = "openjarvis/pearl-miner:<pinned-ref>"
model = "pearl-ai/Llama-3.3-70B-Instruct-pearl"
gateway_port = 8337
gateway_metrics_port = 8339
vllm_port = 8000
gpu_memory_utilization = 0.9
max_model_len = 8192
pearld_rpc_url = "http://localhost:44107"
pearld_rpc_user = "rpcuser"
pearld_rpc_password_env = "PEARLD_RPC_PASSWORD" # name of env var, not the secret
hf_token_env = "HF_TOKEN" # name of env var
```
Secrets: env-var *names*, never literal values. Matches OJ's existing convention for cloud API keys.
### 5.2 JarvisConfig field
`core/config.py` adds:
```python
@dataclass(slots=True)
class JarvisConfig:
...
mining: MiningConfig | None = None
```
The TOML loader reads `[mining]`, parses `submit_target` into `SoloTarget | PoolTarget`, validates against the dataclass, surfaces unknown `extra` keys as warnings. Absent section → `mining = None` → zero behavior change.
### 5.3 Runtime sidecar
`~/.openjarvis/runtime/mining.json` (created on `mine start`, removed on `mine stop`):
```json
{
"provider": "vllm-pearl",
"vllm_endpoint": "http://127.0.0.1:8000/v1",
"model": "pearl-ai/Llama-3.3-70B-Instruct-pearl",
"gateway_url": "http://127.0.0.1:8337",
"gateway_metrics_url": "http://127.0.0.1:8339",
"container_id": "abc123...",
"wallet_address": "prl1q...",
"started_at": 1714867200
}
```
Sidecar deliberately omits all secrets and process IDs. `container_id` is the authoritative handle (Docker is the source of truth for liveness); `wallet_address` is captured for drift-detection (config-vs-runtime).
### 5.4 Engine handoff flow
1. `jarvis mine start` → `MinerRegistry.get("vllm-pearl").start(config)`.
2. `VllmPearlProvider.start()` calls `_docker.PearlDockerLauncher.start(config)` and writes the sidecar.
3. `engine/_discovery.py` checks for `mining.json` on every engine lookup. When present, it auto-registers a `vllm` engine instance pointing at `vllm_endpoint`, named `vllm-pearl-mining`, marked default for mining-aware operations.
4. `jarvis ask` and the SDK route to that endpoint transparently. The user's normal inference is the mining work.
The vLLM engine class itself (`engine/openai_compat_engines.py`) is **not modified**. The change to `engine/_discovery.py` is small and additive: it inspects for `mining.json` and registers a derived `vllm` instance pointing at the mining endpoint when the sidecar is present. Absent sidecar → unchanged discovery behavior.
### 5.5 Manual mode
Power users running their own Pearl container skip `jarvis mine start` and write the sidecar themselves via `jarvis mine attach --vllm-endpoint=... --gateway-url=...`. Decouples lifecycle from wiring.
## 6. CLI surface, lifecycle & daemon integration
### 6.1 Subcommands
| Command | Purpose |
|---|---|
| `jarvis mine init` | Interactive: hardware/Docker checks, prompt for wallet + pearld credentials, write `[mining]`, pull/build image. Does NOT start mining. Pre-checks `>=200 GB` free disk. |
| `jarvis mine start` | Launch container via the registered provider, write sidecar, print endpoint info. Idempotent if running. |
| `jarvis mine stop` | Stop container, remove sidecar. Idempotent if not running. |
| `jarvis mine status` | Read sidecar + query gateway `:8339/metrics`. Print `MiningStats`. |
| `jarvis mine doctor` | Capability matrix; every check ✓/✗ with reason. Works in any state. |
| `jarvis mine attach` | Manual mode: write sidecar without launching. |
| `jarvis mine logs [-f]` | Tail container logs through Docker SDK. |
### 6.2 Doctor output (canonical example)
```
$ jarvis mine doctor
Hardware
GPU vendor nvidia ✓
Compute capability sm_90a ✓
VRAM 80 GB ✓ (need ≥ 70 GB for Pearl 70B)
Docker
Daemon running 24.0.7 ✓
GPU runtime nvidia-container-toolkit ✓
Disk
Free in HF cache 312 GB ✓ (need ≥ 200 GB)
Image
openjarvis/pearl-miner:<ref> present (built 2026-04-30) ✓
Pearl node
RPC http://localhost:44107 ✓
Auth ok ✓
Block height 442107 (synced) ✓
Wallet
Address format prl1q... ✓
Provider capability
vllm-pearl SUPPORTED
Session
Sidecar absent (not running)
Container —
```
Each row maps to one check function in `mining/_discovery.py`. Failures print actionable reasons (e.g. `✗ reason: needs sm90, you have sm89 (RTX 4090)`).
### 6.3 Lifecycle states
```
NOT_CONFIGURED → CONFIGURED → STARTING → RUNNING ⇄ STOPPING → STOPPED
↘
FAILED
```
State derivation rules (no separate state file — derived from config + sidecar + container introspection):
- `NOT_CONFIGURED` — no `[mining]` in config
- `CONFIGURED` — config present, no sidecar
- `STARTING` — sidecar with `started_at` < ~30 s ago, container exists but gateway not yet healthy
- `RUNNING` — sidecar present, container running, gateway responding
- `FAILED` — sidecar present, but container exited or gateway failing > threshold
- `STOPPING` — `mine stop` invoked, Docker stop in progress
- `STOPPED` — `mine stop` complete, sidecar removed
### 6.4 Daemon integration: deliberately minimal in v1
- Docker handles container restart via `--restart=unless-stopped`. OJ does not babysit.
- Existing `com.openjarvis.gateway` daemon is unchanged.
- v1.x hook: `MiningTelemetryCollector` (already shipped in v1, unwired) can be added to the gateway as a 30-second-tick async task.
- launchd/systemd installation surface (`jarvis daemon install`) untouched.
### 6.5 Concurrency
POSIX `flock` on `~/.openjarvis/runtime/mining.lock` prevents racing `mine start` invocations.
### 6.6 `jarvis ask` UX hint
When `[mining]` is configured but no sidecar exists, `cli/hints.py` emits one line: `"mining configured but not running — start it with \`jarvis mine start\`"`. One-line UX nudge, no new infrastructure.
## 7. Pearl Docker integration
### 7.1 Realities from inspecting Pearl's repo
- **Build context = entire Pearl monorepo.** Dockerfile copies root `pyproject.toml`/`uv.lock`, `miner/`, `pearl-blake3/`, `py-pearl-mining/`, `zk-pow/`, `plonky2/`. Building requires the full repo.
- **Pearl publishes no registry image as of writing.** README documents only `docker buildx build -t vllm_miner . -f miner/vllm-miner/Dockerfile`.
- **Single container, three ports.** `entrypoint.sh` launches `pearl-gateway` in the background, waits on `:8339/metrics`, then `exec`s `vllm serve`. Ports: `8000` (vLLM), `8337` (miner RPC), `8339` (gateway metrics).
- **Pinned stack inside the image.** CUDA 12.9.1, vLLM 0.20.0+cu129, Python 3.12, `compute_90a/sm_90a`. Set by Pearl, not by us.
- **First-launch cost.** vLLM pulls the 70 B model from HF on first serve (~140 GB). Build itself is 30–60 min on first init.
### 7.2 Hybrid image acquisition
| Mode | Behavior | When |
|---|---|---|
| **Pre-built pull** | OJ `docker pull`s the configured tag if it resolves in a registry | Default once Pearl publishes; users with private registry; CI |
| **Build-from-pin** | OJ git-clones Pearl at a pinned ref into `~/.openjarvis/cache/pearl/`, then `docker buildx build` | v1 default (Pearl publishes nothing today) |
| **BYO image** | User sets `mining.extra.docker_image_tag` to an image they built/pulled themselves | Power users, air-gapped envs |
Selection logic in `_docker.PearlDockerLauncher.ensure_image()`:
1. If `docker_image_tag` resolves locally → use it.
2. Else `docker pull <tag>` → on success, use it.
3. Else if `tag == OJ_DEFAULT_TAG`, fall back to clone-and-build from `PEARL_PINNED_REF`.
4. Else fail with a clear error pointing at `mine doctor`.
### 7.3 Pearl version pinning
`mining/_constants.py`:
```python
PEARL_REPO = "https://github.com/pearl-research-labs/pearl.git"
PEARL_PINNED_REF = "<sha-or-tag>" # bumped per OJ release after rev-testing
PEARL_IMAGE_TAG = f"openjarvis/pearl-miner:{PEARL_PINNED_REF}"
```
OJ release notes call out the Pearl ref shipped. Bumping the ref is its own PR with a documented Pearl-rev workflow.
### 7.4 Container launch shape
Via `docker>=7.0` SDK in `_docker.PearlDockerLauncher.start()`:
```python
container = client.containers.run(
image=PEARL_IMAGE_TAG,
command=[
config.extra["model"],
"--host", "0.0.0.0",
"--port", str(config.extra["vllm_port"]),
"--gpu-memory-utilization", str(config.extra["gpu_memory_utilization"]),
"--enforce-eager",
"--max-model-len", str(config.extra.get("max_model_len", 8192)),
],
name="openjarvis-pearl-miner",
detach=True,
auto_remove=False,
restart_policy={"Name": "unless-stopped"},
device_requests=[ DeviceRequest(count=-1, capabilities=[["gpu"]]) ],
shm_size="8g",
network_mode="host",
volumes={
str(Path.home() / ".cache/huggingface"): {
"bind": "/root/.cache/huggingface",
"mode": "rw",
},
},
environment={
"PEARLD_RPC_URL": config.extra["pearld_rpc_url"],
"PEARLD_RPC_USER": config.extra["pearld_rpc_user"],
"PEARLD_RPC_PASSWORD": os.environ[config.extra["pearld_rpc_password_env"]],
"PEARLD_MINING_ADDRESS": config.wallet_address,
"HF_TOKEN": os.environ.get(config.extra.get("hf_token_env", "HF_TOKEN"), ""),
"MINER_RPC_TRANSPORT": "tcp",
},
)
```
### 7.5 Trade-offs called out
- **`network_mode="host"`** because pearld's RPC at `http://localhost:44107` lives on the host. A user-defined Docker network adds setup steps with no real isolation benefit on a single-tenant miner box. Pragmatism > purity. Note: host networking has Linux semantics; macOS/Windows Docker handle it differently. Acceptable for v1 since H100/H200 + nvidia-container-toolkit constrains the deployment to Linux anyway.
- **`auto_remove=False`** so a crashed container stays around for `jarvis mine logs` post-mortem.
- **HF cache mounted from host.** 140 GB weight download is one-time, survives container restarts, visible to other tools.
- **Secrets via env-var names**, never persisted in the container image, the sidecar, or Docker labels.
### 7.6 Wallet handling boundary
OJ never sees Pearl mnemonic seeds, never imports Oyster keys, never signs Pearl transactions. Only Pearl-secret OJ touches is the pearld RPC password (passed through container env, sourced by name from host env). Mining address is public — fine in plaintext config.
### 7.7 Image lifecycle UX
- `jarvis mine init` triggers `ensure_image()`, streams build/pull output through the CLI with a clear time estimate (`"Building Pearl miner image — first run takes ~45 min on a fast machine"`).
- `jarvis mine doctor` reports `image: present (tag, age, sha)` or `image: missing (run mine init)`.
- `jarvis mine prune` (v1.x) cleans old `openjarvis/pearl-miner:*` tags. Manual `docker image rm` works in v1.
## 8. Telemetry hooks & v2 fee/pool seams
### 8.1 Telemetry — read surface
Pearl's container exposes `:8339/metrics` (Prometheus exposition format). v1 reads only this endpoint. Deeper RPC introspection via `:8337` deferred to v2.
### 8.2 Adapter and metric mapping
`mining/vllm_pearl.py::_parse_gateway_metrics()` translates Prometheus lines to `MiningStats`. Metric names are TBD on implementation — verified against captured fixture `tests/mining/fixtures/gateway_metrics_sample.txt`. Expected mapping (fallback: zero-fill any missing field, log a one-shot warning):
| `MiningStats` field | Likely Pearl metric (verify on implementation) |
|---|---|
| `shares_submitted` | `pearl_gateway_shares_submitted_total` |
| `shares_accepted` | `pearl_gateway_shares_accepted_total` |
| `blocks_found` | `pearl_gateway_blocks_found_total` |
| `hashrate` | derived rate of `shares_submitted_total` |
| `uptime_seconds` | `process_start_time_seconds` |
| `last_share_at` | `pearl_gateway_last_share_timestamp` |
| `last_error` | derived from `pearl_gateway_errors_total` deltas |
### 8.3 Collection cadence
- **v1: on-demand only.** `jarvis mine status` makes one HTTP GET per call (~10 ms). No background polling.
- **v1.x: `MiningTelemetryCollector` lit up in the gateway daemon.** The class is shipped in v1 but unwired. v1.x adds a periodic asyncio task; same `MiningStats` schema, same gateway endpoint. Zero API churn.
### 8.4 Intelligence-Per-Watt extension
The `telemetry/store.py` schema gains a nullable `mining_session_id` column on inference rows:
- Tagged when an inference goes through the Pearl-mining endpoint; null otherwise.
- Untagged rows behave exactly as today — zero impact on the non-mining path.
- `jarvis telemetry stats --mining` (v1.x) joins to the latest `MiningStats` snapshot and reports `tokens / share`, `joules / share`, `est. PRL / kWh`.
v1 ships the column and the no-op join path. v1.x lights up the reporting. This is the metric the IPW thesis genuinely cares about.
### 8.5 v2 fee/pool seams (three concrete, no more)
**1. `submit_target` parsed into a tagged union; only one variant works.** `SoloTarget` accepted at runtime in v1; `PoolTarget` raises `NotImplementedError("pool support is v2 — track openjarvis#XYZ")`. Reachable only by users who edit their config to opt in.
**2. `fee_bps` / `fee_payout_address` plumbed; zero-valued in v1.** `MiningStats.fees_owed = 0` and `MiningStats.payout_target = "solo"` always in v1. Schema is real; values are zero. No migration in v2.
**3. `mining/pools/` module location reserved.** Empty in v1 except for an `__init__.py` whose docstring says the location is reserved for v2 `PoolClient` work. The v1 spec **does not** define a `PoolClient` ABC — predicting the v2 API precisely creates migration debt. The v2 spec writes against an empty slot.
### 8.6 What v1 deliberately does not lock in
- Pool protocol (PPLNS / PPS / SOLO+ / custom)
- Custody model (escrow / trustless split-coinbase / settlement contract)
- OJ pool URL, share format, share difficulty
- KYC / TOS / payout thresholds
### 8.7 Custody anti-goal
v1 must not introduce any code path where OJ accepts custody of, signs, or routes Pearl funds. Closest v1 comes is reading `wallet_address` (public) and passing it through to the container. v2 revisits.
### 8.8 Single-session assumption (called out, not seamed)
v1 assumes one mining session per host (one sidecar). Multi-GPU / multi-worker fanout is v2+. Sidecar would become a list or directory.
## 9. Failure handling & test strategy
### 9.1 Principles
1. **Fail loud, don't auto-heal.** Docker handles container restarts; `mine doctor` surfaces what's wrong. OJ does not retry mining work, restart pearld, or paper over crashes.
2. **`mine doctor` is the canonical failure surface.** Every failure mode below maps to one or more rows in doctor output.
3. **Sidecar is authoritative; config is intent.** Drift surfaces as a warning, not a crash.
### 9.2 Failure mode matrix
| Failure | v1 behavior | Surface |
|---|---|---|
| Image missing | `mine start` errors with "run `mine init` to build/pull" | `mine doctor: image: missing` |
| GPU not reachable in container | Docker error with `nvidia-container-toolkit` hint | `mine doctor: docker.gpu_runtime: ✗` |
| Disk too low | `mine init` pre-checks `shutil.disk_usage`; errors if < 200 GB free | `mine doctor: disk_free: ✗` |
| vLLM model load fails (HF auth, OOM, model not found) | Container exits; `mine status` reports `FAILED` with `last_error` from `docker logs` tail | `mine status` + `mine logs` |
| pearl-gateway can't reach pearld | `:8339/metrics` exposes the error; adapter populates `MiningStats.last_error` | `mine status: last_error` |
| Container crashes mid-run | Docker `--restart=unless-stopped` restarts; `mine status` shows brief `STARTING` → `RUNNING` | self-healing, logged |
| Stale sidecar (container died, sidecar not cleaned) | `mine start` validates `container_id`; if Docker says it's gone, removes sidecar and proceeds | one-line warning |
| Concurrent `mine start` | POSIX `flock` on `~/.openjarvis/runtime/mining.lock`; second invocation errors clearly | clear message |
| Already-running `mine start` | Idempotent: detect via sidecar + container introspection, print status, exit 0 | informational |
| Wallet/config drift | Sidecar carries wallet from start time; `mine status` cross-checks and warns on mismatch | warning, not auto-restart |
| User edits `submit_target = "pool:..."` in v1 | `start()` raises `NotImplementedError` with tracking issue link | clear error |
| Pearl protocol upgrade (block format / metric names change) | Adapter zero-fills with one-shot warning. `mine doctor` does a best-effort check: it reads the `image: openjarvis/pearl-miner:<ref>` Docker label and compares against `PEARL_PINNED_REF` baked into the OJ release; mismatch surfaces a warning. **OJ does not poll Pearl's GitHub at runtime.** | warning + spec'd Pearl-rev workflow |
| Inference quality regression from NoisyGEMM | **Out of v1 scope to detect.** Documented risk; v1.x may add automated drift detection. | docs only |
### 9.3 Test strategy
Hard constraint: **OJ's CI has no H100, no GPU, no Pearl image, no pearld.** Almost everything must be testable without those.
| Layer | Pattern | Marker | Runs in CI? |
|---|---|---|---|
| `MiningCapabilities.detect()` matrix | Pure unit, parametrized over synthetic `HardwareInfo` | unmarked | yes |
| `MiningConfig` parsing (TOML → dataclass, including `submit_target` tagged-union) | Unit, golden TOML fixtures | unmarked | yes |
| Docker launch shape | `unittest.mock.patch("docker.from_env")`; assert `containers.run(...)` kwargs | unmarked | yes |
| Gateway metrics adapter | `tests/mining/fixtures/gateway_metrics_sample.txt`, parse + assert `MiningStats` | unmarked | yes |
| Sidecar lifecycle (write/read/stale-cleanup, `flock` acquisition) | `tmp_path`, real filesystem, real `flock` | unmarked | yes |
| CLI smoke | Click `CliRunner`, mocked `MiningProvider` | unmarked | yes |
| Container start/stop with real Docker daemon | Real Docker, swap Pearl image for tiny stub `alpine`-based image opening the right ports | new `docker` marker | optional in CI |
| End-to-end mining (real container, real pearld, real shares) | Real H100 + pearld testnet + pinned Pearl image | `live and nvidia and slow` | **no** — manual pre-release smoke |
**New pytest marker.** `docker` registered alongside `live`, `cloud`, `nvidia`, etc. in `pyproject.toml`. CI matrix optionally runs `-m "docker and not live"` on a Docker-enabled runner.
**Conftest hygiene.** `tests/conftest.py`'s autouse fixture clears `MinerRegistry`. `mining/__init__.py`'s `ensure_registered()` survives the autouse clear via `MinerRegistry.contains(...)`.
**Captured Prometheus fixture.** Real metrics output from a Pearl gateway run, committed to the repo. Pins the metric-name assumptions and is the canary if Pearl renames metrics.
### 9.4 What v1 deliberately does not test
- Mining throughput/economics on a real H100 (Pearl's CI tests their kernels)
- Inference quality drift from NoisyGEMM (out of v1 scope)
- Pool share submission paths (v2 spec)
- Apple Silicon paths (Spec B)
## 10. Documentation deliverables (part of this spec)
- `docs/user-guide/mining.md` — user-facing: prerequisites, init flow, doctor output reading guide, `mine status` interpretation, deliberately-unsupported list (Mac, AMD, sm89, non-vLLM engines)
- `docs/development/mining.md` — for contributors: `MiningProvider` ABC, registry pattern, how to add a new provider (Spec B is the canonical worked example)
- One paragraph in `CLAUDE.md` under "Architecture" pointing future-Claude at `mining/` as a sibling subsystem with its own optional-deps discipline
- `REVIEW.md` gets a new bullet under registry compliance specifically calling out `MinerRegistry`
## 11. Open items to resolve at implementation time
1. **Pearl gateway metric names.** Verify the actual exposition labels by capturing `:8339/metrics` from a running Pearl gateway. Update the adapter mapping and commit the fixture.
2. **`PEARL_PINNED_REF`.** Pick a specific commit/tag at the start of implementation. Document the rev-bump workflow.
3. **The `pearl-ai/Llama-3.3-70B-Instruct-pearl` HF model.** Confirm it exists and is gated/ungated; document HF auth requirements.
4. **Pearl Taproot address regex.** Confirm the prefix and length for `mine doctor`'s address-format check.
5. **Pearl `:8337` miner RPC TCP port behavior.** Confirm `MINER_RPC_TRANSPORT=tcp` works as documented and binds to `0.0.0.0` not just `127.0.0.1` inside the host network namespace.
6. **OJ default Docker image tag.** Decide whether to publish to GHCR/Docker Hub once we have a build, or leave users on build-from-pin. Likely v1.x.
7. **Wallet generation hand-off (v1.x).** Decide whether `mine init` shells out to Pearl's `oyster` for users who want guidance, or stays paste-only.
8. **Telemetry schema migration approach.** Adding the nullable `mining_session_id` column to `telemetry/store.py` is a SQLite schema change. Decide between (a) `ALTER TABLE` on first start with a guarded `PRAGMA user_version` bump, (b) per-query `try/except` on the column, or (c) creating a sidecar table joined on inference id. Confirm what convention OJ already uses for `telemetry/` schema evolution before picking; default lean is (a).
## 12. Cross-references
- **[Spec B — Apple Silicon enablement](2026-05-05-apple-silicon-pearl-mining-design.md)** — separate effort tracking the Pearl-side and OJ-side work to make Apple Silicon a registered `MiningProvider`. Spec A is engine-agnostic by design; Spec B drops in via `MinerRegistry` without modifying anything in this spec.
- **Pearl repo:** [`pearl-research-labs/pearl`](https://github.com/pearl-research-labs/pearl) — referenced sub-paths: `miner/vllm-miner/`, `miner/pearl-gemm/`, `miner/pearl-gateway/`, `miner/vllm-miner/Dockerfile`, `miner/vllm-miner/entrypoint.sh`.
- **Pearl paper:** [Proof-of-Useful-Work via matrix multiplication (arXiv:2504.09971)](https://arxiv.org/abs/2504.09971).
- **OJ contributing guide:** `docs/development/contributing.md` — registry pattern, `_stubs.py` / `_discovery.py` conventions, `ensure_registered()` discipline, optional-deps soft-import pattern. All followed in this spec.
## 13. Implementation plan
The implementation plan for Spec A is a separate document, written via the `superpowers:writing-plans` skill after this design is approved by the user. It will decompose section 4–9 above into ordered, independently-reviewable steps and call out which steps can be parallelized.
docs/design/2026-05-05-vllm-pearl-mining-integration-plan.md
+3558
View File
File diff suppressed because it is too large Load Diff
docs/development/contributing.md
-1
View File
@@ -413,4 +413,3 @@ policy:
if the component requires new packages
See the [registry pattern](#registry-pattern) section above for complete examples.
docs/development/mining-nvidia-validation.md
+289
View File
@@ -0,0 +1,289 @@
# NVIDIA Pearl Mining Validation Runbook
This runbook is the release gate for the v1 `vllm-pearl` provider. Unit tests
prove OpenJarvis wiring; this validates that a real H100/H200 host can mine
through Pearl and serve inference through OpenJarvis.
## Required Host
Run this on a Linux machine with:
- NVIDIA H100 or H200 GPU, compute capability 9.0, at least 70 GB VRAM
- Current NVIDIA driver with CUDA container support
- Docker 24+ and `nvidia-container-toolkit`
- At least 200 GB free disk
- Reachable `pearld` JSON-RPC endpoint
- Pearl payout address beginning with `prl1q` or `prl1p`
- Hugging Face access to `pearl-ai/Llama-3.3-70B-Instruct-pearl`
The validated H100 configuration uses `gpu_memory_utilization = 0.96` with
`max_model_len = 8192`. Lower memory utilization can fail during vLLM startup
because the Pearl 70B mining model leaves too little KV cache at 8k context.
Do not run this on macOS, Apple Silicon, AMD, RTX 4090, or CPU-only hosts.
Those are separate providers.
## Wallet Address Setup
Create the wallet from the Pearl repo root:
```bash
./bin/oyster -u rpcuser -P rpcpass --create
```
If you choose the optional public-data encryption prompt, Oyster will require
that public passphrase on startup via `--walletpass`. Keep private and public
passphrases out of shell history where possible.
Start Oyster:
```bash
./bin/oyster \
-u rpcuser \
-P rpcpass \
--walletpass '<public-wallet-passphrase-if-configured>' \
&
```
Then generate a mining address through the wallet RPC:
```bash
./bin/prlctl \
--wallet \
--skipverify \
-u rpcuser \
-P rpcpass \
-s localhost:44207 \
getnewaddress
```
Notes:
- `--wallet` is required. Without it, `prlctl` talks to `pearld` instead of
Oyster and may look for `Pearld/pearld.conf`.
- Use `-s localhost:44207`, not `-s https://localhost:44207`. `prlctl` expects
host and port, not a URL.
- `--skipverify` is acceptable for this local validation flow unless you have
configured the Oyster RPC certificate path.
- If a mnemonic has been pasted into logs, chat, or a PR, discard that wallet
and create a fresh one before mining.
## Environment
```bash
git checkout feat/mining-spec-a-only
uv sync --extra dev --extra mining-pearl-vllm
export PEARLD_RPC_PASSWORD='<pearld-rpc-password>'
export HF_TOKEN='<huggingface-token>'
```
Confirm host prerequisites:
```bash
nvidia-smi
docker info
docker run --rm --gpus all nvidia/cuda:12.9.1-base-ubuntu24.04 nvidia-smi
df -h ~/.cache
```
Expected:
- `nvidia-smi` shows H100 or H200.
- Docker can run a CUDA container with GPU access.
- `~/.cache` or the Hugging Face cache volume has at least 200 GB free.
On shared hosts, select only idle GPUs during `mine init`:
```bash
uv run jarvis mine init --cuda-visible-devices 0
```
This writes `[mining.extra].cuda_visible_devices`. `mine start` passes that
device list to Docker and sets `CUDA_VISIBLE_DEVICES` /
`NVIDIA_VISIBLE_DEVICES` inside the container. Omit the option only on a
dedicated host where the miner may use all GPUs.
## Configure Mining
Run:
```bash
uv run jarvis mine doctor
```
Before config exists, `doctor` should show hardware and Docker as OK, and
Pearl node / wallet as unconfigured.
Then initialize:
```bash
uv run jarvis mine init
```
Use:
- Wallet: the user's Pearl `prl1q...` or `prl1p...` address
- `pearld` URL: usually `http://localhost:44107`
- RPC user: configured `pearld` user, often `rpcuser`
- Password env: `PEARLD_RPC_PASSWORD`
- Model: `pearl-ai/Llama-3.3-70B-Instruct-pearl`
- Image: default unless validating a custom Pearl image
- CUDA devices: an idle GPU ID such as `0` on shared hosts
Expected:
- `[mining]` and `[mining.extra]` are written to config.
- Image resolves locally, pulls, or builds from the pinned Pearl ref.
- First build may take 30-60 minutes.
Run `doctor` again:
```bash
uv run jarvis mine doctor
```
Expected:
- Hardware OK
- Docker OK
- Disk OK
- Pearl node RPC OK and synced
- Wallet format OK
- `vllm-pearl SUPPORTED`
- Sidecar absent
## Start Mining
```bash
uv run jarvis mine start
```
Expected:
- Docker container `openjarvis-pearl-miner` starts.
- `~/.openjarvis/runtime/mining.json` is written.
- Sidecar contains `vllm_endpoint`, `gateway_url`, `gateway_metrics_url`, and
`container_id`.
Inspect:
```bash
docker ps --filter name=openjarvis-pearl-miner
cat ~/.openjarvis/runtime/mining.json
uv run jarvis mine logs --tail 200
uv run jarvis mine status
```
Expected:
- Container is running.
- vLLM is listening on the configured port, default `8000`.
- Pearl gateway metrics are available on the configured metrics port, default
`8339`.
- `mine status` exits 0 and prints `provider: vllm-pearl`.
## Verify OpenJarvis Inference Uses Mining Endpoint
Run:
```bash
uv run jarvis mine doctor
uv run jarvis ask "Say hello in one sentence."
```
Expected:
- `doctor` shows sidecar present.
- Engine discovery registers `vllm-pearl-mining`.
- The prompt completes through the Pearl/vLLM endpoint.
- Container logs show vLLM activity during the prompt.
If inference succeeds but mining stats stay zero, continue to the Pearl
network checks below; vLLM serving alone is not enough to prove mining.
## Verify Pearl Network Submission
Check gateway metrics directly:
```bash
curl -fsS http://127.0.0.1:8339/metrics | tee /tmp/pearl-gateway-metrics.txt
uv run jarvis mine status
```
Expected:
- Metrics endpoint returns Prometheus text.
- If Pearl exposes share counters, `mine status` maps them correctly.
- If metric names differ, attach `/tmp/pearl-gateway-metrics.txt` to the PR and
update `src/openjarvis/mining/_metrics.py`.
Check `pearld` connectivity using the same RPC configuration used by mining:
```bash
curl --user "rpcuser:${PEARLD_RPC_PASSWORD}" \
--data-binary '{"jsonrpc":"1.0","id":"oj","method":"getblockchaininfo","params":[]}' \
-H 'content-type: text/plain;' \
http://127.0.0.1:44107
```
Expected:
- `blocks` and `headers` are present.
- Node is synced or close enough for mining validation.
Proof of actual earning requires a successful accepted share/block and wallet
credit. Depending on Pearl network difficulty, this may take longer than the
smoke test window. Record:
- Runtime duration
- `mine status` before and after
- Gateway metrics snapshot
- Relevant container log tail
- Wallet balance / transaction evidence if a reward lands
## Stop And Cleanup
```bash
uv run jarvis mine stop
docker ps --filter name=openjarvis-pearl-miner
test ! -e ~/.openjarvis/runtime/mining.json
```
Expected:
- Container stops.
- Sidecar is removed.
- `jarvis ask` no longer routes through `vllm-pearl-mining` unless another
mining sidecar is attached.
## Pass Criteria
The NVIDIA provider is considered proven when all are true:
- `mine doctor` reports supported on H100/H200.
- `mine init` resolves/builds the Pearl image.
- `mine start` launches the container and writes the sidecar.
- OpenJarvis inference succeeds through `vllm-pearl-mining`.
- Pearl gateway metrics are reachable and `mine status` parses them.
- `pearld` accepts the miner's network path.
- At least one accepted share/block is observed, or a documented Pearl
maintainer confirmation says the observed gateway state is sufficient proof
of live mining.
## Failure Artifacts
For any failure, collect:
```bash
uv run jarvis mine doctor
uv run jarvis mine status || true
uv run jarvis mine logs --tail 300 || true
docker inspect openjarvis-pearl-miner || true
curl -fsS http://127.0.0.1:8339/metrics || true
nvidia-smi
docker info
```
Attach outputs to the implementation PR or follow-up issue. Do not paste
`PEARLD_RPC_PASSWORD`, wallet seed material, or Hugging Face tokens.
docs/development/mining.md
+84
View File
@@ -0,0 +1,84 @@
# Adding a Mining Provider
The `openjarvis.mining` subsystem follows the same registry pattern as engines,
agents, tools, memory, and channels. New mining paths should be provider
modules, not special cases in the CLI or engine layer.
## Provider Contract
Every provider implements `openjarvis.mining.MiningProvider`:
- `detect(hw, engine_id, model)` is pure capability detection. It must not
start subprocesses, hit the network, or mutate state.
- `start(config)` owns provider lifecycle setup and writes the mining sidecar
when it changes inference routing.
- `stop()` tears down provider-owned processes or containers.
- `is_running()` answers from provider-owned state.
- `stats()` returns `MiningStats` using the provider's most stable telemetry
surface.
Register providers through `MinerRegistry` and expose idempotent
`ensure_registered()`:
```python
from openjarvis.core.registry import MinerRegistry
def ensure_registered() -> None:
if not MinerRegistry.contains("my-provider"):
MinerRegistry.register_value("my-provider", MyProvider)
```
`tests/conftest.py` clears registries between tests, so test fixtures and CLI
entry points should call `ensure_registered()` before relying on a provider.
## Optional Dependencies
Provider dependencies belong in scoped extras:
- `mining-pearl-vllm` for the NVIDIA/vLLM Docker provider
- Future Apple work should use a separate extra such as `mining-pearl-metal`
or `mining-pearl-cpu`
Avoid a generic `mining-pearl` extra until there is a shared dependency set
that every provider actually needs.
## Sidecar Contract
The runtime sidecar lives at `~/.openjarvis/runtime/mining.json`. Engine
handoff is data-driven:
- If the sidecar has `vllm_endpoint`, engine discovery registers
`vllm-pearl-mining`.
- If a future provider mines alongside the user's normal engine, it should omit
`vllm_endpoint`; engine discovery will ignore it.
Do not branch on `provider == "vllm-pearl"` in generic code. Branch on sidecar
shape or provider capability.
## Apple Silicon Handoff
The Apple Silicon effort should add its own provider module and reuse:
- `MiningProvider`
- `MinerRegistry`
- `MiningConfig`
- `MiningStats`
- `Sidecar`
- `jarvis mine doctor` capability iteration
That work should not need to rewrite the NVIDIA provider, CLI group, telemetry
collector, or engine sidecar handoff.
## NVIDIA Release Gate
The NVIDIA provider is not considered economically proven until the H100/H200
runbook passes on real hardware. See
[`mining-nvidia-validation.md`](./mining-nvidia-validation.md) for the required
commands, artifacts, and pass criteria.
## Model Enablement
New Pearl-compatible language models are tracked separately from provider
support. See [`pearl-model-enablement.md`](./pearl-model-enablement.md) for the
conversion and validation checklist.
docs/development/pearl-model-enablement.md
+93
View File
@@ -0,0 +1,93 @@
# Pearl Model Enablement
This page tracks the work required to make a new Hugging Face model mineable
through Pearl's vLLM miner and OpenJarvis.
OpenJarvis can point `vllm-pearl` at a model id, but a raw Hugging Face model is
not enough. The Pearl vLLM plugin expects a Pearl-compatible quantized model
whose metadata marks mining layers for 7-bit NoisyGEMM and non-mining layers
for the vanilla Pearl GEMM path.
## Target Models
| Raw model | Planned Pearl model | Status | Tracking |
|---|---|---|---|
| `Qwen/Qwen3.5-9B` | `pearl-ai/Qwen3.5-9B-pearl` | Planned | [#316](https://github.com/open-jarvis/OpenJarvis/issues/316) |
| `Qwen/Qwen3.6-27B` | `pearl-ai/Qwen3.6-27B-pearl` | Planned | [#317](https://github.com/open-jarvis/OpenJarvis/issues/317) |
| `google/gemma-4-E4B-it` | `pearl-ai/Gemma-4-E4B-it-pearl` | Planned | [#318](https://github.com/open-jarvis/OpenJarvis/issues/318) |
| `google/gemma-4-31B-it` | `pearl-ai/Gemma-4-31B-it-pearl` | Planned | [#319](https://github.com/open-jarvis/OpenJarvis/issues/319) |
The current validated model remains:
```text
pearl-ai/Llama-3.3-70B-Instruct-pearl
```
## Enablement Checklist
1. Reproduce the current Llama Pearl model recipe.
- Record the compressed-tensors config.
- Record which linear layers are 7-bit mining layers.
- Record which layers are 8-bit non-mining layers.
- Record calibration data and SmoothQuant settings, if used.
2. Convert the target model.
- Start with `Qwen/Qwen3.5-9B`; it is the smallest target.
- Generate Pearl-compatible quantized weights and metadata.
- Publish under the planned `pearl-ai/*-pearl` id or a staging namespace.
3. Validate the Pearl vLLM plugin path.
- Model loads in Pearl's `vllm-miner` container.
- vLLM registers Pearl's quantization plugin.
- Mining layers use int7 NoisyGEMM.
- Non-mining layers use int8 vanilla Pearl GEMM.
- Text generation works with mining enabled and disabled.
4. Validate chain integration.
- `pearld` is reachable.
- `pearl-gateway` receives work.
- NoisyGEMM submits candidate proofs.
- Gateway reports metrics.
- `jarvis mine status` parses those metrics.
5. Promote the model in OpenJarvis.
- Change its registry status from `planned` to `validated`.
- Set measured VRAM and context defaults.
- Add the model to user docs.
- Attach validation logs to the PR.
## OpenJarvis Registry
Model support metadata lives in:
```text
src/openjarvis/mining/_models.py
```
`jarvis mine models` renders that registry. Planned models are visible to users
but blocked by capability detection until the Pearl model artifact and H100/H200
validation exist.
## Acceptance Criteria
A model is `validated` only when all of these pass on real hardware:
- `jarvis mine init --model <pearl-model-id>`
- `jarvis mine start`
- `curl http://127.0.0.1:8000/v1/models`
- `jarvis ask "Say hello in one sentence."`
- `jarvis mine status`
- `jarvis mine validate-model --model <pearl-model-id> --allow-planned --prompt
"Say hello in one sentence." --output <artifact>.json`
- Pearl gateway metrics show the mining path is active.
- No block/share submission errors appear in gateway or miner logs.
Do not mark a model validated based only on vLLM load success. It must exercise
Pearl's NoisyGEMM and submission path.
## Tracking
Use the `Pearl Model Validation` GitHub issue template for each candidate model.
The issue should hold the quantization recipe, hardware details, command output,
metrics excerpts, and the PR that changes the model status to `validated`.
Attach the JSON artifact from `jarvis mine validate-model --output` to the issue.
docs/user-guide/cli.md
+29
View File
@@ -199,6 +199,35 @@ jarvis model pull qwen3:8b
---
## `jarvis pearl`
Access Pearl's native node, wallet, and RPC tools from the OpenJarvis CLI.
```bash
jarvis pearl doctor
jarvis pearl node -- <pearld args>
jarvis pearl wallet -- <oyster args>
jarvis pearl ctl -- <prlctl args>
jarvis pearl address
```
All Pearl wrapper commands use the `jarvis pearl <command>` shape. The
pass-through commands map to Pearl's native binaries:
| OpenJarvis command | Pearl binary | Use |
|--------------------|--------------|-----|
| `jarvis pearl doctor` | n/a | Check whether `pearld`, `oyster`, and `prlctl` are discoverable |
| `jarvis pearl node` | `pearld` | Run the Pearl full node |
| `jarvis pearl wallet` | `oyster` | Run the Oyster wallet daemon |
| `jarvis pearl ctl` | `prlctl` | Query Pearl node or wallet RPC |
| `jarvis pearl address` | `prlctl --wallet getnewaddress` | Generate a wallet address from Oyster |
Use `PEARL_HOME=/path/to/pearl` or `--pearl-home /path/to/pearl` if Pearl's
`bin/` directory is not on `PATH`. See the [Pearl CLI guide](pearl.md) for
examples.
---
## `jarvis memory`
Manage the document memory store for retrieval-augmented generation.
docs/user-guide/mining-apple-silicon.md
+162
View File
@@ -0,0 +1,162 @@
# Mining Pearl on Apple Silicon (and other CPU hosts)
OpenJarvis can mine the [Pearl](https://github.com/pearl-research-labs/pearl) chain
on Apple Silicon Macs (M1/M2/M3/M4) using the `cpu-pearl` provider. **This is
v1**: decoupled CPU mining. Your existing local LLM workflow (Ollama, MLX-LM,
llama.cpp, vLLM) is untouched; mining runs in the background as a separate
process.
## Honest expectations
**Hashrate on Apple Silicon CPU is far below what an H100 produces with
Pearl's `vllm-miner`.** A rough rule of thumb (subject to network difficulty):
- M2 Max / M4 Max: ≪ 1 share per second at typical mainnet difficulty
- H100 with `vllm-miner`: meaningfully higher, plus the mining work is
amortized over real LLM inference
If you want to mine for yield, this isn't the path. If you want to participate
in the network from the hardware you own, with no special hardware purchase,
this is the path.
An experimental `apple-mps-pearl` provider is available for developers. It
uses PyTorch MPS for the NoisyGEMM matmuls, while transcript hashing and proof
construction still run on CPU. This proves the Apple-GPU path can produce
validator-accepted `PlainProof`s, but it is not yet the high-performance Metal
kernel path.
## Prerequisites
- macOS arm64 (M1, M2, M3, M4) — or Linux x86_64 / aarch64
- Python 3.12 (`brew install python@3.12` or use `uv venv --python 3.12`)
- Rust toolchain (`brew install rust` or `curl https://sh.rustup.rs -sSf | sh`)
- Your own running [`pearld`](https://github.com/pearl-research-labs/pearl#node)
node, RPC reachable on `http://localhost:44107`
- A Pearl Taproot wallet address from `oyster` (Pearl's wallet CLI)
- ~1 GB free disk for the Pearl source clone and build artifacts
## Install
```bash
# from your OpenJarvis repo
uv sync --extra mining-pearl-cpu
```
If Pearl wheels are not yet on PyPI (still true as of 2026-05-05), `uv sync`
succeeds but doesn't install the actual Pearl Python packages. Build/install
them from a local Pearl checkout:
```bash
cd /path/to/pearl/py-pearl-mining
maturin build --release
uv pip install target/wheels/py_pearl_mining-*.whl
uv pip install ../miner/miner-utils ../miner/pearl-gateway ../miner/miner-base
```
## Configure
Create a Pearl wallet and start a synced `pearld` separately using Pearl's
README. Then write OpenJarvis' mining config:
```bash
export PEARLD_RPC_PASSWORD="rpcpass"
jarvis mine init \
--provider cpu-pearl \
--wallet-address "<your-prl1...address>" \
--pearld-rpc-url http://127.0.0.1:44107 \
--pearld-rpc-user rpcuser \
--pearld-rpc-password-env PEARLD_RPC_PASSWORD
```
On Apple Silicon, `--provider auto` chooses `apple-mps-pearl`; use
`--provider cpu-pearl` for the conservative CPU path. The MPS path is
experimental and currently useful for validation/profiling, not revenue.
This writes:
```toml
[mining]
provider = "cpu-pearl"
wallet_address = "prl1..."
submit_target = "solo"
fee_bps = 0
[mining.extra]
pearld_rpc_url = "http://127.0.0.1:44107"
pearld_rpc_user = "rpcuser"
pearld_rpc_password_env = "PEARLD_RPC_PASSWORD"
gateway_host = "127.0.0.1"
gateway_port = 8337
metrics_port = 9109
```
## Run
```bash
jarvis mine doctor # capability matrix
jarvis mine start # launch gateway + miner-loop subprocesses
jarvis mine status # check sidecar + gateway metrics
jarvis mine logs -n 120 # print recent logs
jarvis mine stop # stop mining subprocesses
```
## Reading `mine doctor`
Each row is one check. `✓` means the check passed; `✗` shows the actionable fix.
```
$ jarvis mine doctor
Hardware
GPU vendor apple ✓
Apple chip M2 Max ✓
Pearl install
py-pearl-mining 0.1.0 (cp312-abi3-macos-arm64) ✓
miner-base 0.1.0 ✓
pearl-gateway 0.1.0 ✓
Pearl node
RPC http://localhost:44107 ✓
Block height 442107 (synced) ✓
Wallet
Address format prl1q... ✓
Provider capability
cpu-pearl SUPPORTED (calibrated 0.X share/h on M2 Max)
Notes
- This is decoupled mining: your normal LLM inference is unaffected
- Hashrate is far below H100 mining; see this doc above
- MPS mining: available as experimental apple-mps-pearl
Session
Sidecar absent (not running)
```
## Limitations
- **Windows is not supported in v1.** Pearl's pure-Rust miner builds on
Windows in principle but the cross-platform install path is untested. Use
WSL2 if you must.
- **No coupling to inference yet.** v1 is a separate process; your CPU does
mining, your GPU does inference. They don't share work. v2 changes this.
- **Experimental PyTorch-MPS only.** `apple-mps-pearl` moves the NoisyGEMM
matmuls to MPS but still has CPU readbacks for transcript hashing and proof
construction. Use it for validation and profiling, not revenue expectations.
- **No multi-host pool.** Solo mining only. The pool work is a separate spec.
## Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| `mine doctor` says `Pearl Python packages not installed` | Wheels not built yet | Run `jarvis mine init` |
| `pearl-gateway` log shows `connection refused` to `http://localhost:44107` | `pearld` not running | Start `pearld` per Pearl's README |
| `mine status` shows `last_error: gateway metrics unreachable` | `pearl-gateway` crashed | Check `~/.openjarvis/logs/mining/pearl-gateway.log` |
| Build fails with `error: linker 'cc' not found` | Xcode CLT not installed | `xcode-select --install` |
| `maturin build` complains about `tikv-jemallocator` | macOS SDK too old | Update macOS / Xcode |
For anything not on this list, capture `~/.openjarvis/logs/mining/` and open
an issue at https://github.com/open-jarvis/OpenJarvis/issues.
## What changes in v2 / v3
- **v2:** Optimize the current `apple-mps-pearl` path, then optionally plug it
into MLX-LM or `llama-cpp-python` so inference matmuls become mining work.
- **v3 (only if v2 perf is insufficient):** Native Metal kernel as a Pearl
upstream contribution. No user-visible change other than higher hashrate.
docs/user-guide/mining.md
+133
View File
@@ -0,0 +1,133 @@
# Pearl Mining
OpenJarvis can mine the Pearl Proof-of-Useful-Work chain through local LLM
inference. The primary v1 path supports NVIDIA H100/H200 hosts running vLLM
with Pearl's Docker miner. The consolidated Pearl integration also includes
experimental Apple Silicon and CPU providers through the same `MiningProvider`
registry.
## Prerequisites
| Requirement | v1 expectation |
|---|---|
| GPU | NVIDIA H100 or H200, sm_90a class, at least 70 GB VRAM |
| OS | Linux with `nvidia-container-toolkit` configured |
| Docker | Docker 24+ with GPU runtime access |
| Disk | At least 200 GB free for the 70B model and build cache |
| Pearl node | Reachable `pearld` JSON-RPC endpoint, default `http://localhost:44107` |
| Wallet | Pearl address beginning with `prl1q` or `prl1p` |
The default vLLM config uses `gpu_memory_utilization = 0.96` and
`max_model_len = 8192` for the Pearl 70B mining model on H100/H200 80 GB GPUs.
To generate a wallet address with Pearl's Oyster wallet, run Pearl's wallet
daemon and query it with `prlctl --wallet --skipverify -s localhost:44207
getnewaddress`. Do not reuse a wallet whose mnemonic has been pasted into logs,
chat, or issue trackers.
## Quick Start
```bash
uv sync --extra mining-pearl-vllm
export PEARLD_RPC_PASSWORD=<your-pearld-password>
export HF_TOKEN=<your-huggingface-token>
uv run jarvis mine init
uv run jarvis mine start
uv run jarvis mine status
```
`mine init` writes a `[mining]` config section and resolves the Pearl Docker
image. If Pearl has not published a suitable image for the pinned ref,
OpenJarvis falls back to building from the pinned Pearl source checkout. First
builds can take 30-60 minutes.
On a shared NVIDIA host, restrict the miner to idle GPUs:
```bash
uv run jarvis mine init --cuda-visible-devices 0
```
This writes `[mining.extra].cuda_visible_devices`, which `mine start` passes to
Docker instead of exposing every GPU on the machine.
## Commands
- `jarvis mine models` lists Pearl model support status.
- `jarvis mine doctor` prints hardware, Docker, Pearl node, wallet, provider,
and session checks.
- `jarvis mine init` writes the local mining config and resolves the image.
- `jarvis mine start` launches the Pearl miner container and writes the runtime
sidecar.
- `jarvis mine stop` stops the provider and removes the sidecar.
- `jarvis mine status` reads live gateway metrics.
- `jarvis mine attach` writes a sidecar for a miner you launched manually.
- `jarvis mine logs` prints the Docker container log tail.
- `jarvis mine validate-model` probes the active vLLM miner and gateway before
promoting a planned Pearl model to validated.
## Model Support
Run:
```bash
jarvis mine models
```
OpenJarvis only enables models that have Pearl-compatible quantized artifacts
and real hardware validation. Raw Hugging Face models such as
`Qwen/Qwen3.5-9B` or `google/gemma-4-E4B-it` are not mineable by themselves;
they need corresponding `pearl-ai/*-pearl` variants.
The default validated model is:
```text
pearl-ai/Llama-3.3-70B-Instruct-pearl
```
The Qwen and Gemma targets are tracked in the model registry as planned until
Pearl quantization and H100/H200 validation are complete.
When validating a newly converted Pearl model on a mining host, run:
```bash
jarvis mine validate-model \
--model pearl-ai/Qwen3.5-9B-pearl \
--allow-planned \
--prompt "Say hello in one sentence." \
--output qwen3.5-9b-pearl-validation.json
```
Remove `--allow-planned` only after the model is promoted to validated in the
OpenJarvis registry. Attach the JSON artifact to the validation issue.
## v1 Scope
v1 is solo mining only. OpenJarvis does not take fees, custody funds, generate
wallet keys, run pools, or operate `pearld`. Users provide their own Pearl node
and payout address.
Unsupported in this PR:
- Pool mining and the future 20% OpenJarvis fee model
- AMD GPU mining and non-Pearl backends
- RTX 4090 or other non-Hopper NVIDIA GPUs
- Wallet generation or transaction signing inside OpenJarvis
## Troubleshooting
Run:
```bash
uv run jarvis mine doctor
```
Read the rows top-down. Fix the first failing dependency before retrying
`mine start`. A Mac or AMD machine should fail honestly at provider capability;
those paths are expected to land as separate providers.
## Production Readiness
The NVIDIA path requires one real H100/H200 validation run before it should be
marketed as a proven earning path. The developer runbook is
[`../development/mining-nvidia-validation.md`](../development/mining-nvidia-validation.md).
docs/user-guide/pearl.md
+58
View File
@@ -0,0 +1,58 @@
# Pearl CLI Integration
OpenJarvis includes a thin `jarvis pearl` wrapper for Pearl's native command
line tools. It does not replace Pearl's node or wallet; it makes the common
commands discoverable from the same CLI users use for mining.
## Binary Discovery
`jarvis pearl` looks for `pearld`, `oyster`, and `prlctl` on `PATH`, then under
`$PEARL_HOME/bin`.
```bash
export PEARL_HOME=/path/to/pearl
jarvis pearl doctor
```
## Native Pass-Through
Use pass-through commands when you need the full Pearl surface:
```bash
jarvis pearl node -- --help
jarvis pearl wallet -- --help
jarvis pearl ctl -- --help
```
These map directly to:
| OpenJarvis command | Pearl binary |
|---|---|
| `jarvis pearl node` | `pearld` |
| `jarvis pearl wallet` | `oyster` |
| `jarvis pearl ctl` | `prlctl` |
The command format is always `jarvis pearl <command>`. Pearl-native arguments
go after that command. Use `--` before Pearl arguments when the arguments begin
with dashes and you want to make the pass-through boundary explicit.
## Wallet Address Helper
If Oyster is already running, generate a mining address through wallet RPC:
```bash
jarvis pearl address \
-u rpcuser \
-P rpcpass \
-s localhost:44207
```
The helper uses `prlctl --wallet` and defaults to `--notls`, which matches the
local validation flow. Use `--tls --skipverify` if your Oyster RPC endpoint is
serving TLS with a local certificate.
## Boundary
`jarvis mine` is the OpenJarvis mining lifecycle. `jarvis pearl` is an escape
hatch to Pearl's native node, wallet, and RPC tools. For advanced node or
wallet administration, Pearl's own help output is the source of truth.