[RFC]: Bringing `glob` Dependency In-House

# Bringing the `glob` Dependency In-House

Replacing the external `glob` npm package with stdlib-native equivalents to eliminate a supply-chain risk and align all production and tooling code with stdlib's quality standards (docs, tests, examples, benchmarks, backward compatibility).

## Background

### What `glob` Does

A robust glob implementation in JavaScript that matches files using patterns the shell uses (like stars and question marks). It works by walking the filesystem and applying regex-based matching to paths.
More here: https://github.com/isaacs/node-glob

### Its core dependencies

- **`minimatch`**: Converts the glob strings into regular expressions and performs the actual string matching.
- **`fs.realpath` / `inflight` / `once`**: Various utilities used under the hood to manage concurrent filesystem calls and path resolutions.

### How stdlib uses it

- **50+ files** across: `@stdlib/_tools/pkgs/*`, `@stdlib/_tools/lint/*`, `@stdlib/_tools/static-analysis/*`, and various internal bundling or testing scripts.
- **Only the core discovery functions are used**: `glob( pattern, opts, clbk )` and `glob.sync( pattern, opts )`.
- **Primary use cases**: Finding `package.json` files, source `.js` files, or `.c` files, while explicitly passing an `ignore` option array (e.g., `['node_modules/**', '.git/**']`) to prevent scanning massive dependency directories.
- **No advanced API usage**: None of the files instantiate the `Glob` class directly as an event emitter or use advanced bash features like brace expansion (`{a,b}`).

---

## Proposed Changes

The plan creates **2 new packages** mirroring stdlib's decomposable architecture, with each package being independently consumable.

---

### Component 1: `@stdlib/utils/regexp-from-glob`

> Replaces the `minimatch` npm dependency (transitive via `glob`).

A utility to convert standard shell wildcard patterns into native JavaScript `RegExp` objects.

**Scope**: Strictly scoped to the wildcards `stdlib` actually uses (`*`, `**`, `?`). It will not support brace expansions or extglobs to remain intentionally minimal and maintain high execution speed.

```
@stdlib/utils/regexp-from-glob/
├── lib/
│ ├── index.js # re-export main
│ └── main.js # regexpFromGlob(str) → RegExp
├── test/
│ └── test.js
├── benchmark/
│ └── benchmark.js
├── docs/
│ ├── repl.txt
│ └── types/
│ ├── index.d.ts
│ └── test.ts
├── examples/
│ └── index.js
├── README.md
└── package.json
```

**API**:
```js
var regexpFromGlob = require( '@stdlib/utils/regexp-from-glob' );

var re = regexpFromGlob( '**/*.js' );
// returns RegExp

re.test( 'lib/index.js' ); // => true
re.test( 'lib/index.json' ); // => false
```

---

### Component 2: `@stdlib/fs/glob`

> The core package — the direct replacement for `require('glob')`.

A filesystem traversal utility that applies the generated glob regexes to discover files. This is the main package all `_tools` files will switch to.

```
@stdlib/fs/glob/
├── lib/
│ ├── index.js # re-export main
│ ├── main.js # async implementation
│ ├── sync.js # sync implementation
│ ├── walk.js # internal DFS/BFS directory walker
│ └── validate.js # options validation
├── test/
│ ├── test.js # async tests
│ ├── test.sync.js # sync tests
│ └── test.walk.js # directory walker tree-pruning tests
├── benchmark/
│ ├── benchmark.js # benchmark: async traversal
│ └── benchmark.sync.js # benchmark: sync traversal
├── docs/
│ ├── repl.txt
│ └── types/
│ ├── index.d.ts
│ └── test.ts
├── examples/
│ └── index.js
├── README.md
└── package.json
```

**API** (drop-in compatible with current usage):
```js
var glob = require( '@stdlib/fs/glob' );

var opts = {
'cwd': __dirname,
'ignore': [ 'node_modules/**' ],
'realpath': true
};

// Async usage
glob( '**/*.js', opts, function onGlob( error, matches ) {
if ( error ) {
console.error( error );
return;
}
console.dir( matches );
});

// Sync usage
var matches = glob.sync( '**/*.js', opts );
```

**Key implementation details**:
- Uses `@stdlib/fs/read-dir` and standard `fs.stat` under the hood.
- **Tree-Pruning (Crucial for perf)**: The internal `walk.js` algorithm must evaluate the `ignore` option *before* descending into a directory.
- Options support: `cwd` (defaults to `process.cwd()`), `ignore` (array of globs), and `realpath` (boolean).
- Returns arrays strictly normalized, preventing duplicates.
- **No external dependencies** (uses existing `stdlib` utilities).

---

### Component 3: Migration — Updating All Consumers

This is the mechanical bulk change, done after the new packages are created and verified.

#### [MODIFY] All 50+ files using `require( 'glob' )`

The change is a single-line replacement per file:

```diff
-var glob = require( 'glob' );
+var glob = require( '@stdlib/fs/glob' );
```

**Migration strategy** (phased to reduce risk):

| Phase | Scope | Files |
|-------|-------|-------|
| **Phase 1** | `@stdlib/_tools/pkgs/*` (find, deps, clis, etc.) | ~15 files |
| **Phase 2** | `@stdlib/_tools/static-analysis/*` (sloc-glob, etc.) | ~10 files |
| **Phase 3** | `@stdlib/_tools/lint/*` (filenames, pkg-json, etc.) | ~20 files |
| **Phase 4** | Remaining bundles and scripts | ~10 files |

Each phase follows the same process:
1. Run `find` + `sed` to replace `require( 'glob' )` → `require( '@stdlib/fs/glob' )`.
2. Run existing internal tool tests for the affected script area.
3. Verify output matches the original glob dependency.

#### Uninstall the dependancy

After all phases are complete:

```
npm uninstall glob
```

---

## Verification Plan

### Automated Tests

#### 1. Unit tests for `@stdlib/utils/regexp-from-glob`

```bash
make TESTS_FILTER=".*/utils/regexp-from-glob/.*" test
```

Tests should cover:
- Core wildcards (`*`, `**`, `?`).
- Escaping behavior for dot `.`, plus `+`, parentheses `()`, and brackets `[]`.
- Prefix, suffix, and exact match conditions.

#### 2. Unit tests for `@stdlib/fs/glob`

```bash
make TESTS_FILTER=".*/fs/glob/.*" test
```

Tests should cover:
- Standard sync and async pattern matching against a mock filesystem.
- `ignore` arrays effectively pruning traversal.
- `cwd` option shifts the search base.
- `realpath` correctly resolves to absolute paths.
- Error handling (e.g., trying to read restricted directories gracefully).

#### 3. Regression tests for migrated modules

Run the full test suite for each phase to ensure the tools operate as expected:

```bash
# Phase 1:
make TESTS_FILTER=".*/_tools/pkgs/.*" test

# Phase 2:
make TESTS_FILTER=".*/_tools/static-analysis/.*" test

# Phase 3:
make TESTS_FILTER=".*/_tools/lint/.*" test

# Phase 4: Full internal tools suite
make test
```

### Manual Verification

After migration, verify the tools perform correctly on the monorepo:

1. **Verify Package Discovery**:
```bash
node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli
```
→ Verify: Outputs all valid `stdlib` sub-packages identically to previous runs, without diving into `node_modules`.

2. **Verify SLOC execution**:
```bash
node ./lib/node_modules/@stdlib/_tools/static-analysis/js/sloc-glob/bin/cli
```
→ Verify: Accurately computes lines of code for the workspace.

3. **Performance Check**:
Time the execution before and after the swap.
```bash
time node ./lib/node_modules/@stdlib/_tools/pkgs/find/bin/cli
```
→ Verify: Our native, tree-pruned walker executes in equivalent or faster time than the external dependency.