mirror of
https://github.com/garrytan/gbrain.git
synced 2026-08-14 08:53:22 +00:00
v0.42.67.0 fix(build): force LF for shell scripts and route package.json checks through bash (#3506)
* v0.42.67.0 fix(build): force LF for shell scripts and route package.json checks through bash Two independent defects left `bun run test`, `verify`, `ci:local` and `test:e2e` dead on Windows. All four dispatch through bash. First, every tracked *.sh is checked out with CRLF. The committed blobs are clean LF; system-level core.autocrlf=true rewrites them on checkout, and a strict bash then dies at run-unit-parallel.sh line 23 with "$'\r': command not found". A root .gitattributes pinning `*.sh text eol=lf` overrides autocrlf regardless of the contributor's git config. Second, 33 package.json scripts invoked `scripts/foo.sh` directly, which bun cannot exec via shebang on Windows. They now go through `bash`, matching the 11 that already did; all 59 tracked *.sh files are bash-shebanged (52 `#!/usr/bin/env bash`, 7 `#!/bin/bash`), so the change is uniform. The five `scripts/*.ts` entries still run under bun. Measured on this base, `bun run verify` goes from pass=1 fail=31 to pass=25 fail=7. Every one of the baseline's 29 `command not found: scripts/...` errors is gone; those were the shebang defect, and they account for the measured delta. The line-ending defect is verified structurally rather than by that number, because the bash on PATH for this measurement tolerates CR and so cannot exhibit it: under the new attribute all 59 tracked *.sh check out LF-only (0/59 carry a CR byte, against 59/59 before), and `git add --renormalize .` is a no-op, confirming the index was always correct and only the working tree was wrong. Zero content churn. All 7 residual failures also fail on the pristine baseline: four exceed the harness's 120s cap (standalone `bun run typecheck` exits 0), and check:wasm, check:skill-brain-first and check:resolver are pre-existing content or environment issues. check:resolver is not even a shell script. No behavior change on Linux or macOS. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs: sync docs to v0.42.67.0 CONTRIBUTING.md gains a Windows section: the `.gitattributes` LF pin makes a fresh clone correct with no extra steps, working copies cloned earlier need a one-time `git rm --cached -r . -q && git reset --hard`, and new shell-script checks must be registered as `bash scripts/<name>.sh`. docs/TESTING.md records the shell-dispatch convention alongside the command-tier table, and notes that the table's wallclock figures are Mac numbers: on Windows `check:privacy`, `check:test-names`, `check:test-isolation` and `typecheck` can exceed run-verify-parallel.sh's 120s per-check cap while passing on Linux and macOS. It also flags that the Cygwin bash shipped with Git for Windows tolerates CRLF where a strict bash does not, so a green local run is not evidence that a script is CRLF-clean. CHANGELOG.md's itemized list covers both doc updates. `bun run build:llms` regenerates byte-identical bundles: docs/TESTING.md is linked rather than inlined, so llms.txt / llms-full.txt do not move. `bun test test/build-llms.test.ts` passes 12/12. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
daragao3
co-authored by
Claude Opus 5
Claude Opus 5
parent
e58abd652c
commit
3df20f9f18
@@ -0,0 +1,16 @@
|
||||
# Line-ending policy.
|
||||
#
|
||||
# Shell scripts MUST be checked out with LF endings on every platform.
|
||||
# Git for Windows installs with `core.autocrlf=true` by default, which
|
||||
# rewrites LF -> CRLF on checkout. A strict bash (WSL, Linux CI, macOS)
|
||||
# then chokes on the trailing CR:
|
||||
#
|
||||
# scripts/run-unit-parallel.sh: line 23: $'\r': command not found
|
||||
# scripts/run-unit-parallel.sh: line 24: set: pipefail : invalid option name
|
||||
# scripts/run-unit-parallel.sh: line 32: syntax error near unexpected token `$'{\r''
|
||||
#
|
||||
# That silently disabled `bun run test`, `bun run verify`, `bun run ci:local`
|
||||
# and `bun run test:e2e` for Windows contributors, since all four dispatch
|
||||
# through bash. `eol=lf` pins the checkout regardless of the user's
|
||||
# core.autocrlf setting.
|
||||
*.sh text eol=lf
|
||||
@@ -2,6 +2,45 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.67.0] - 2026-07-28
|
||||
|
||||
**If you develop GBrain on Windows, the test and check commands now actually run. Until this release they were quietly doing almost nothing.**
|
||||
|
||||
`bun run test`, `bun run verify`, `bun run ci:local` and `bun run test:e2e` all hand off to shell scripts, and on Windows that hand-off was broken in two separate places. The commands did not stop with an obvious error. They reported a result, so a run could look finished when barely any of the checks had actually inspected anything. On a clean Windows clone, `bun run verify` got 1 check to pass and 31 to fail. It now gets 25 to pass and 7 to fail, and none of the 7 are caused by this change.
|
||||
|
||||
The first problem was line endings. Git for Windows installs with `core.autocrlf=true`, which rewrites shell scripts to Windows line endings when you clone or check out. Bash refuses to run those, so a script died on its second line before doing any work. The scripts stored in the repository were always correct; only the copy on your disk was wrong. A new `.gitattributes` pins every `.sh` file to Unix line endings at checkout, no matter how your Git is configured.
|
||||
|
||||
The second problem was how the checks were started. Thirty three of them pointed straight at a `.sh` file. On macOS and Linux the shell reads the `#!/usr/bin/env bash` line at the top of the script and runs it correctly. Bun on Windows does not do that, so those commands failed the moment they were called. They now go through `bash` explicitly, the same way the other eleven were already written.
|
||||
|
||||
Nothing changes for macOS and Linux. No stored file content moves, and no check behaves differently on those platforms.
|
||||
|
||||
## To take advantage of v0.42.67.0
|
||||
|
||||
Only Windows contributors need to do anything, and only once. `.gitattributes` applies at checkout time, so shell scripts already sitting on your disk keep their old line endings until you refresh them.
|
||||
|
||||
1. **Refresh the working copy** from the repository root:
|
||||
```bash
|
||||
git rm --cached -r . -q
|
||||
git reset --hard
|
||||
```
|
||||
2. **Confirm bash can read the scripts:**
|
||||
```bash
|
||||
bash -n scripts/run-unit-parallel.sh
|
||||
```
|
||||
Silence means it worked. `$'\r': command not found` means step 1 did not take effect.
|
||||
3. **Run the gate:**
|
||||
```bash
|
||||
bun run verify
|
||||
```
|
||||
|
||||
### Itemized changes
|
||||
|
||||
- New root `.gitattributes` pins `*.sh text eol=lf`, so shell scripts check out with Unix line endings regardless of the contributor's `core.autocrlf` setting. All 59 tracked `.sh` files were already stored with Unix endings, so `git add --renormalize .` reports nothing to do and no stored content changes.
|
||||
- `package.json` now routes the remaining 33 `.sh` check commands through `bash`, matching the 11 that already did. Every tracked `.sh` file carries a bash shebang (52 `#!/usr/bin/env bash` and 7 `#!/bin/bash`), so the treatment is uniform across all of them.
|
||||
- The five `scripts/*.ts` entries still run under bun and are untouched.
|
||||
- `CONTRIBUTING.md` gains a Windows section covering the one-time working-copy refresh and the `bash scripts/<name>.sh` convention for new checks.
|
||||
- `docs/TESTING.md` records how the test commands dispatch through bash, and notes that three tree-walking checks plus `typecheck` can exceed the 120s per-check cap on Windows while passing on Linux and macOS.
|
||||
|
||||
## [0.42.66.1] - 2026-07-27
|
||||
|
||||
### Fixed
|
||||
|
||||
@@ -11,6 +11,28 @@ bun test
|
||||
|
||||
Requires Bun 1.0+.
|
||||
|
||||
### Windows
|
||||
|
||||
`bun run test`, `verify`, `ci:local` and `test:e2e` all dispatch through bash, so
|
||||
the shell scripts under `scripts/` must be checked out with Unix line endings.
|
||||
The root `.gitattributes` pins `*.sh text eol=lf`, which overrides the
|
||||
`core.autocrlf=true` that Git for Windows installs by default. A fresh clone is
|
||||
correct with no extra steps.
|
||||
|
||||
If you cloned before that pin existed, your working copy still has the old
|
||||
Windows line endings and bash will fail with `$'\r': command not found`. Refresh
|
||||
it once, from the repository root:
|
||||
|
||||
```bash
|
||||
git rm --cached -r . -q
|
||||
git reset --hard
|
||||
bash -n scripts/run-unit-parallel.sh # silence means bash can read the scripts
|
||||
```
|
||||
|
||||
Every `check:*` entry in `package.json` invokes its script as `bash scripts/<name>.sh`
|
||||
rather than relying on the shebang, because bun on Windows cannot exec a `.sh`
|
||||
directly. Keep that prefix when you add a new shell-script check.
|
||||
|
||||
## Project structure
|
||||
|
||||
```
|
||||
|
||||
@@ -1,5 +1,25 @@
|
||||
# TODOS
|
||||
|
||||
## v0.42.67.0 follow-ups (Windows build tooling)
|
||||
|
||||
Filed as follow-ups from v0.42.67.0 (`.gitattributes` LF pin for `*.sh` +
|
||||
`bash` prefix on the 33 `package.json` check commands). Both items are newly
|
||||
observable: before that release these checks never executed on Windows at all,
|
||||
so nothing about their runtime was measurable.
|
||||
|
||||
- [ ] **P2 — three guard scripts exceed the 120s `run-verify-parallel.sh` cap on Windows.**
|
||||
With the dispatch fixed, `bun run verify` on Windows gets 25 passes and 7 failures, and
|
||||
`check:privacy`, `check:test-names` and `check:test-isolation` are timeouts rather than
|
||||
real failures (they pass on Linux and macOS well inside the cap). They walk the tree with
|
||||
per-file shell loops, which is far slower under Windows process creation. Either raise the
|
||||
cap for these three, or replace the per-file loop with a single `grep -r` pass. Same cap
|
||||
swallows `typecheck`, though standalone `bun run typecheck` exits 0.
|
||||
- [ ] **P3 — `check:wasm` cannot create its `node_modules` symlink on Windows.**
|
||||
`scripts/check-wasm-embedded.sh` fails with `ln: failed to create symbolic link
|
||||
'/tmp/gbrain-wasm-check.XXXX/node_modules': No such file or directory`. Unprivileged
|
||||
Windows accounts cannot create symlinks without developer mode. Consider a junction, a
|
||||
copy, or skipping the check with a clear message when symlink creation is unavailable.
|
||||
|
||||
## community fix-wave follow-ups (filed v0.42.60.0)
|
||||
|
||||
- [x] **P2 — cherry-pick #2112's uncovered doctor.ts hunk.** Fix-wave A (#2820) superseded
|
||||
|
||||
@@ -19,6 +19,29 @@ Seven test command tiers, each with a clear scope:
|
||||
| `bun run test:e2e` | Real Postgres E2E. Requires Docker + `DATABASE_URL`. Sequential. | ~5-10min | Pre-ship; nightly. |
|
||||
| `bun run check:all` | The historical pre-check scripts (22, chained sequentially in package.json). Overlaps `verify` heavily but is NOT a superset — `verify`'s `CHECKS` array in `scripts/run-verify-parallel.sh` (~30 entries incl. typecheck) is the authoritative gate; `check:all` keeps a few local-only extras (trailing-newline, exports-count, no-legacy-getconnection). | ~10s | Local-only sweep for the extras. |
|
||||
|
||||
### Shell dispatch and Windows
|
||||
|
||||
All four of `test`, `verify`, `ci:local` and `test:e2e` hand off to shell scripts
|
||||
under `scripts/`, so every `check:*` entry in `package.json` invokes its script as
|
||||
`bash scripts/<name>.sh` instead of relying on the shebang — bun on Windows cannot
|
||||
exec a `.sh` directly. Add a new shell-script check with that same prefix. The
|
||||
`scripts/*.ts` entries run under bun and take no prefix.
|
||||
|
||||
The scripts must also be on disk with Unix line endings. A strict bash (WSL, Linux
|
||||
CI, macOS) rejects CRLF and dies on the script's first meaningful line; the Cygwin
|
||||
bash that ships with Git for Windows tolerates it, so a green local run is not by
|
||||
itself evidence that a script is CRLF-clean.
|
||||
The root `.gitattributes` pins `*.sh text eol=lf`, which overrides the
|
||||
`core.autocrlf=true` default that Git for Windows installs. Working copies cloned
|
||||
before that pin need a one-time `git rm --cached -r . -q && git reset --hard` to
|
||||
pick it up; see the Windows section of `CONTRIBUTING.md`.
|
||||
|
||||
Wallclock figures in the table above are from a Mac dev box. Windows is
|
||||
substantially slower because each check pays full process-creation cost, and three
|
||||
tree-walking checks (`check:privacy`, `check:test-names`, `check:test-isolation`)
|
||||
plus `typecheck` can exceed the 120s per-check cap in `run-verify-parallel.sh`
|
||||
there even though they pass on Linux and macOS.
|
||||
|
||||
### CI vs local: intentionally divergent file sets
|
||||
|
||||
- **CI matrix** (`.github/workflows/test.yml`) runs `scripts/test-shard.sh` across 10 matrix shards partitioned by weight-aware LPT bin-packing (`scripts/sharding.ts`) and INCLUDES `*.slow.test.ts` (the two outlier slow files run as dedicated jobs alongside the matrix). CI EXCLUDES `*.serial.test.ts` from the shards and runs them in a dedicated job via `bun run test:serial`, one bun process per file — keeping serial files out of the shard processes is what preserves the `mock.module` quarantine (a top-level mock in one file leaks into every other file sharing its process). `bun run verify` gets its own job too. CI is the ground truth for "did everything pass."
|
||||
|
||||
+34
-34
@@ -42,20 +42,20 @@
|
||||
"eval:autocut": "bun test test/search/autocut-eval.test.ts",
|
||||
"test:full": "bun run verify && bash scripts/run-unit-parallel.sh && bun run test:slow && ([ -n \"$DATABASE_URL\" ] && bash scripts/run-e2e.sh || echo '[test:full] skipped E2E (no DATABASE_URL); run docker-compose -f docker-compose.ci.yml up + bun run test:e2e to include' 1>&2)",
|
||||
"verify": "bash scripts/run-verify-parallel.sh",
|
||||
"check:source-config-leak": "scripts/check-source-config-leak.sh",
|
||||
"check:no-pii-agent-voice": "scripts/check-no-pii-in-agent-voice.sh",
|
||||
"check:synthetic-corpus-privacy": "scripts/check-synthetic-corpus-privacy.sh",
|
||||
"check:system-of-record": "scripts/check-system-of-record.sh",
|
||||
"check:admin-scope-drift": "scripts/check-admin-scope-drift.sh",
|
||||
"check:cli-exec": "scripts/check-cli-executable.sh",
|
||||
"check:all": "scripts/check-privacy.sh && scripts/check-proposal-pii.sh && scripts/check-test-real-names.sh && scripts/check-jsonb-pattern.sh && scripts/check-source-id-projection.sh && scripts/check-source-config-leak.sh && scripts/check-progress-to-stdout.sh && scripts/check-no-tracked-symlinks.sh && scripts/check-no-legacy-getconnection.sh && scripts/check-test-isolation.sh && scripts/check-trailing-newline.sh && scripts/check-wasm-embedded.sh && scripts/check-exports-count.sh && scripts/check-admin-build.sh && scripts/check-admin-scope-drift.sh && scripts/check-cli-executable.sh && scripts/check-skill-brain-first.sh && scripts/check-operations-filter-bypass.sh && scripts/check-gateway-routed-no-direct-anthropic.sh && scripts/check-worker-pool-atomicity.sh && scripts/check-key-files-current-state.sh && scripts/check-no-double-retry.sh && scripts/check-batch-audit-site.sh",
|
||||
"check:gateway-routed": "scripts/check-gateway-routed-no-direct-anthropic.sh",
|
||||
"check:worker-pool-atomicity": "scripts/check-worker-pool-atomicity.sh",
|
||||
"check:doc-history": "scripts/check-key-files-current-state.sh",
|
||||
"check:source-config-leak": "bash scripts/check-source-config-leak.sh",
|
||||
"check:no-pii-agent-voice": "bash scripts/check-no-pii-in-agent-voice.sh",
|
||||
"check:synthetic-corpus-privacy": "bash scripts/check-synthetic-corpus-privacy.sh",
|
||||
"check:system-of-record": "bash scripts/check-system-of-record.sh",
|
||||
"check:admin-scope-drift": "bash scripts/check-admin-scope-drift.sh",
|
||||
"check:cli-exec": "bash scripts/check-cli-executable.sh",
|
||||
"check:all": "bash scripts/check-privacy.sh && bash scripts/check-proposal-pii.sh && bash scripts/check-test-real-names.sh && bash scripts/check-jsonb-pattern.sh && bash scripts/check-source-id-projection.sh && bash scripts/check-source-config-leak.sh && bash scripts/check-progress-to-stdout.sh && bash scripts/check-no-tracked-symlinks.sh && bash scripts/check-no-legacy-getconnection.sh && bash scripts/check-test-isolation.sh && bash scripts/check-trailing-newline.sh && bash scripts/check-wasm-embedded.sh && bash scripts/check-exports-count.sh && bash scripts/check-admin-build.sh && bash scripts/check-admin-scope-drift.sh && bash scripts/check-cli-executable.sh && bash scripts/check-skill-brain-first.sh && bash scripts/check-operations-filter-bypass.sh && bash scripts/check-gateway-routed-no-direct-anthropic.sh && bash scripts/check-worker-pool-atomicity.sh && bash scripts/check-key-files-current-state.sh && bash scripts/check-no-double-retry.sh && bash scripts/check-batch-audit-site.sh",
|
||||
"check:gateway-routed": "bash scripts/check-gateway-routed-no-direct-anthropic.sh",
|
||||
"check:worker-pool-atomicity": "bash scripts/check-worker-pool-atomicity.sh",
|
||||
"check:doc-history": "bash scripts/check-key-files-current-state.sh",
|
||||
"check:resolver": "bun src/cli.ts check-resolvable --strict --skills-dir skills/",
|
||||
"check:skill-brain-first": "scripts/check-skill-brain-first.sh",
|
||||
"check:wasm": "scripts/check-wasm-embedded.sh",
|
||||
"check:newlines": "scripts/check-trailing-newline.sh",
|
||||
"check:skill-brain-first": "bash scripts/check-skill-brain-first.sh",
|
||||
"check:wasm": "bash scripts/check-wasm-embedded.sh",
|
||||
"check:newlines": "bash scripts/check-trailing-newline.sh",
|
||||
"test:e2e": "bash scripts/run-e2e.sh",
|
||||
"test:slow": "bash scripts/run-slow-tests.sh",
|
||||
"test:heavy": "bash scripts/run-heavy.sh",
|
||||
@@ -65,27 +65,27 @@
|
||||
"ci:local:diff": "bash scripts/ci-local.sh --diff",
|
||||
"ci:select-e2e": "bun run scripts/select-e2e.ts",
|
||||
"typecheck": "tsc --noEmit",
|
||||
"check:jsonb": "scripts/check-jsonb-pattern.sh",
|
||||
"check:search-path": "scripts/check-search-path.sh",
|
||||
"check:no-double-retry": "scripts/check-no-double-retry.sh",
|
||||
"check:batch-audit-site": "scripts/check-batch-audit-site.sh",
|
||||
"check:worker-lock-renewal-shape": "scripts/check-worker-lock-renewal-shape.sh",
|
||||
"check:source-id-projection": "scripts/check-source-id-projection.sh",
|
||||
"check:privacy": "scripts/check-privacy.sh",
|
||||
"check:proposal-pii": "scripts/check-proposal-pii.sh",
|
||||
"check:eval-glossary": "scripts/check-eval-glossary-fresh.sh",
|
||||
"check:test-names": "scripts/check-test-real-names.sh",
|
||||
"check:progress": "scripts/check-progress-to-stdout.sh",
|
||||
"check:no-tracked-symlinks": "scripts/check-no-tracked-symlinks.sh",
|
||||
"check:exports-count": "scripts/check-exports-count.sh",
|
||||
"check:admin-build": "scripts/check-admin-build.sh",
|
||||
"check:admin-embedded": "scripts/check-admin-embedded.sh",
|
||||
"check:test-isolation": "scripts/check-test-isolation.sh",
|
||||
"check:fuzz-purity": "scripts/check-fuzz-purity.sh",
|
||||
"check:operations-filter-bypass": "scripts/check-operations-filter-bypass.sh",
|
||||
"check:fixture-privacy": "scripts/check-fixture-privacy.sh",
|
||||
"check:jsonb": "bash scripts/check-jsonb-pattern.sh",
|
||||
"check:search-path": "bash scripts/check-search-path.sh",
|
||||
"check:no-double-retry": "bash scripts/check-no-double-retry.sh",
|
||||
"check:batch-audit-site": "bash scripts/check-batch-audit-site.sh",
|
||||
"check:worker-lock-renewal-shape": "bash scripts/check-worker-lock-renewal-shape.sh",
|
||||
"check:source-id-projection": "bash scripts/check-source-id-projection.sh",
|
||||
"check:privacy": "bash scripts/check-privacy.sh",
|
||||
"check:proposal-pii": "bash scripts/check-proposal-pii.sh",
|
||||
"check:eval-glossary": "bash scripts/check-eval-glossary-fresh.sh",
|
||||
"check:test-names": "bash scripts/check-test-real-names.sh",
|
||||
"check:progress": "bash scripts/check-progress-to-stdout.sh",
|
||||
"check:no-tracked-symlinks": "bash scripts/check-no-tracked-symlinks.sh",
|
||||
"check:exports-count": "bash scripts/check-exports-count.sh",
|
||||
"check:admin-build": "bash scripts/check-admin-build.sh",
|
||||
"check:admin-embedded": "bash scripts/check-admin-embedded.sh",
|
||||
"check:test-isolation": "bash scripts/check-test-isolation.sh",
|
||||
"check:fuzz-purity": "bash scripts/check-fuzz-purity.sh",
|
||||
"check:operations-filter-bypass": "bash scripts/check-operations-filter-bypass.sh",
|
||||
"check:fixture-privacy": "bash scripts/check-fixture-privacy.sh",
|
||||
"check:conversation-parser": "bun src/cli.ts eval conversation-parser test/fixtures/conversation-formats/all.jsonl --no-llm",
|
||||
"check:source-scope-onboard": "scripts/check-source-scope-onboard.sh",
|
||||
"check:source-scope-onboard": "bash scripts/check-source-scope-onboard.sh",
|
||||
"postinstall": "bun run scripts/postinstall.ts",
|
||||
"prepublish:clawhub": "bun run build:all",
|
||||
"publish:clawhub": "clawhub package publish . --family bundle-plugin"
|
||||
@@ -146,7 +146,7 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.66.1",
|
||||
"version": "0.42.67.0",
|
||||
"overrides": {
|
||||
"@hono/node-server": "^2.0.5",
|
||||
"fast-uri": "^3.1.4",
|
||||
|
||||
Reference in New Issue
Block a user