garrytan / garrytan/gstack

macOS: 3 tests in test/gstack-design-detect.test.ts fail deterministically — $TMPDIR vs its realpath (/var -> /private/var)

Open Beginner friendly
#2,841 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
133k
Forks
19.9k
Avg merge
18h 46m
Merged PRs (30d)
26

Description

## Summary

Three tests in `test/gstack-design-detect.test.ts` fail deterministically on macOS because the test
builds its expected path from `$TMPDIR` while the code under test has already resolved that path to
its real location. On macOS `$TMPDIR` is `/var/folders/…` and `/var` is a symlink to `/private/var`,
so the two strings differ by that one prefix even though they name the same directory.

This is not flaky — it fails on gstack's own serial flaky-retry as well.

## Environment

- gstack `1.84.1.0` (`71f6048e`)
- macOS 26.6.2, arm64
- bun 1.3.14

## Failing tests

From `bun run test:free` (shard 5/7 failed with 3; the `GSTACK_FREE_RETRY_FLAKY=1` serial retry
failed with 4):

- `probe > ignored rules and files are the union of config.json and config.local.json; malformed config is reported`
- `scan: option-like bases and page-controlled inline ignores > from HOME (no repository) HOME-rooted installs are READY, HOME files are refused as targets, and a dump still scans without inline ignores`
- `scan: option-like bases and page-controlled inline ignores > the whole-scan budget stops a huge target set instead of grinding batch after batch` (retry only)
- `install: the one download gstack makes, after consent > downloads from a mirror, verifies the checksum, installs under IMPECCABLE_HOME, receipts the fetch first, and the probe finds it`

## Reproduction

```
$ echo $TMPDIR
/var/folders/qp/…/T/
$ cd "$TMPDIR" && pwd -P
/private/var/folders/qp/…/T
```

Any assertion that joins `$TMPDIR` into an expected path and compares it against output the
implementation has realpath'd will differ by the `/private` prefix.

## Confirmation

Running that test file with a `TMPDIR` that is already its own realpath:

```
TMPDIR="$(cd "$TMPDIR" && pwd -P)" bun test test/gstack-design-detect.test.ts
```

| | before | after |
|---|---|---|
| `test/gstack-design-detect.test.ts` | 62 pass, 3 fail | **65 pass, 0 fail** |

No other test file changes result. I specifically A/B'd the path-security tests, since a naive
"resolve all temp paths" change could plausibly weaken them — they are unaffected:

| | before | after |
|---|---|---|
| `browse/test/path-validation.test.ts` | 29 pass, 0 fail | 29 pass, 0 fail |
| `browse/test/temp-dirs.test.ts` | 7 pass, 0 fail | 7 pass, 0 fail |
| `browse/test/data-platform.test.ts` | 15 pass, 0 fail | 15 pass, 0 fail |

(One caveat worth knowing if you reproduce this: `browse/test/path-validation.test.ts` and
`temp-dirs.test.ts` resolve paths relative to cwd and treat the temp dir as allowlisted, so running
them from a checkout that itself lives under `/tmp` produces three unrelated failures. Clone under
`$HOME` to reproduce cleanly — that cost me an hour of chasing a phantom.)

## Suggested fix

Resolve `$TMPDIR` to its real path where the tests construct expected paths (or have the tests
compare realpath-to-realpath rather than raw strings). Happy to send a PR if you'd like it in a
particular shape — I didn't want to guess at your preference between fixing it in the individual
assertions vs. a shared test helper.

## Why this matters beyond one machine

CI presumably runs on Linux, where `/tmp` has no such symlink, so this is invisible there and hits
every macOS user. On my side it was surfaced by an auto-updater that runs the free suite as a gate
before accepting a new gstack — with this bug, every release rolls back on macOS.

Contributor guide

Open the contributing guide

Research direction

Start in test/gstack-design-detect.test.ts and inspect the expected paths built from $TMPDIR alongside the realpath-resolved output. Run TMPDIR="$(cd "$TMPDIR" && pwd -P)" bun test test/gstack-design-detect.test.ts, then make the three macOS failures pass without changing the path-security behavior; the file should report 65 passing tests and no failures.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, typescript
Domain
testing-qa
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
85/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.