`apm compile` is very slow on large monorepos with scoped `applyTo` patterns (quadratic placement cost)

Problem

apm compile's ContextOptimizer gets slow on large monorepos once a project has multiple scoped (non-universal) applyTo patterns, each matching a large, overlapping set of directories, the shape of a real monorepo with per-module conventions. I hit this on an internal monorepo with roughly 159,000 directories and 294,000 files, where compile time made the command impractical to use.

Digging into ContextOptimizer, four costs stand out:

• _file_matches_pattern and _safe_recursive_glob resolve every file path independently per pattern instead of reusing a memoized relative path.
• _calculate_distribution_score recomputes the depth mean inside its variance loop (O(n²)) and rescans _directory_cache for a count that's just its length.
• _is_hierarchically_covered, _find_minimal_coverage_placement, and _is_instruction_relevant resolve paths that are already canonical _directory_cache keys.
• _optimize_single_point_placement rescans every matching directory per candidate instead of short-circuiting on the first miss.

Most of this is just removing redundant work. One change does affect behavior, see below.

Behavior change: symlinked files

Before, _file_matches_pattern called file_path.resolve() before checking relative_to(self.base_dir). For a file that's actually a symlink into somewhere outside the project root, resolve() follows it to the real path, which then falls outside base_dir and gets excluded. So symlinked-in files were silently treated as not part of the project.

The fix drops that resolve() call in favor of a memoized relative path computed straight from the walk, which already doesn't follow symlinks. Symlinked files now match applyTo patterns based on where they appear in the project tree, not where they physically live. I think that's the more useful behavior for placement, but it's a real change and worth scoping explicitly rather than assuming it away.

Evidence / reproduction

Before/after benchmark on synthetic monorepo-shaped projects: https://github.com/sekosen/apm-placement-benchmark

Before = latest main (f8df1b75), after = perf/placement-large-tree-scaling:

Alternatives considered

• Caching at a higher level, for example skipping ContextOptimizer entirely for large trees. Rejected, that changes behavior instead of just cutting redundant computation.
• Leaving it as is. Not viable at monorepo scale, compile time grows superlinearly with directory count.

Expected benefit

Compile time on large monorepos drops from roughly quadratic to near-linear. It passes on all tests relevant to this change in the existing unit/integration suite (a few unrelated failures exist independent of this branch), but the suite doesn't seem to exercise symlinked-in files, so it never caught the behavior change above.

Related

The implementation already exists and got reviewed and benchmarked in #2878, closed as part of the issue-first intake reset. It wasn't actually behavior-neutral once you count the symlink change, so that's worth scoping explicitly here instead of assumed away. Once this issue's scope is approved, I'll ask on #2878 for it to be reopened, per CONTRIBUTING.md.

Start with the ContextOptimizer entry point used by apm compile, then trace _file_matches_pattern, _safe_recursive_glob, _calculate_distribution_score, and the placement helpers named in the issue. Run the existing unit and integration suite alongside the linked benchmark, and verify that the final behavior explicitly covers symlinked-in files and large overlapping directory patterns.