bazelbuild / bazelbuild/rules_swift

Feature request: swift.use_swiftinterface_for_caching for large monorepos

Open
#1,881 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Starlark
Stars
353
Forks
172
Avg merge
23h 14m
Merged PRs (30d)
8

Description

## Summary

Proposing an opt-in feature `swift.use_swiftinterface_for_caching` that lets Swift compile actions key their action cache on transitive deps' `.swiftinterface` files (rather than `.swiftmodule`) when those deps are built with `library_evolution`. Swiftmodule files remain available to the compiler through Bazel's `unused_inputs_list` mechanism, so compile behavior is unchanged — only the cache key changes.

Implementation, end-to-end verification, and fallback behavior in PR #1880 .

## Motivation

I'm aware the recommended posture is to enable `library_evolution` per-library rather than globally, and that's the right default for a ruleset serving both small integrations and large codebases. But at scale — million-line-plus mixed Swift/Objective-C monorepos with hundreds of Swift modules — the trade-off flips, and both **global `library_evolution`** and **swiftinterface-based cache keys** become effectively required. Here's why:

**Without swiftinterface caching, `library_evolution`'s cache benefit is invisible.**

Today every downstream Swift compile action's cache key hashes each transitive dependency's `.swiftmodule`. A `.swiftmodule` embeds serialized SIL, `internal` symbol tables, private inline bodies — anything the compiler might need across module boundaries. A purely-internal change (adding a `private` helper, tweaking a non-`public` body) rewrites the swiftmodule and evicts caches for every downstream module. `library_evolution` doesn't change that: it produces a `.swiftinterface` that stays stable across internal changes, but that swiftinterface never enters the cache key.

Net effect: teams enable `library_evolution` expecting downstream builds to be insulated from internal churn, pay the extra frontend cost, and don't see the corresponding cache-hit-rate improvement. So they either turn it off or restrict it to a handful of "framework boundary" libraries — and the promise of ABI-stable cache boundaries never materializes for the ~99% of intra-monorepo dependencies where it would have paid off most.

**Why "just enable it per-library" doesn't scale.**

In a monorepo, "public API" means "consumed by another module in the same repo," not "shipped externally." The ABI contract is between an organization and itself. Per-library opt-in imposes ongoing review overhead across hundreds of `BUILD` files with no corresponding safety benefit — CI catches ABI regressions the same commit, not weeks later at a downstream integrator. What large teams need is: `library_evolution` on across the tree, `emit_swiftinterface` on, and the cache-key benefit auto-derived from that.

## Production experience

We've been running an equivalent patch on rules_swift 3.1.2 in production for a mixed Swift/Objective-C monorepo (1M+ LOC of Swift, hundreds of Swift modules). Two observed effects:

1. **Downstream cache reuse on internal changes.** Commits that touch only `private` bodies in an upstream module now produce identical `.swiftinterface`; downstream modules — including hundreds of transitive consumers — hit the action cache instead of recompiling. Before the patch, the same commit would invalidate the entire reverse dependency closure. This is what we care about most.

2. **Global `library_evolution` becomes viable.** Once cache reuse pays for it, `swift.enable_library_evolution` on globally does not regress CI or local build times — the per-module frontend cost is dwarfed by the compile actions we no longer re-run.

We're now rebasing onto rules_swift 4.0.0-rc4 for a Bazel 9.2 migration and re-porting the patch. Rather than continue maintaining it downstream, I'd like to upstream it.

## Design (details in the PR)

Opt-in, off by default:

- New feature `swift.use_swiftinterface_for_caching`.
- Enabled → `.swiftinterface` (`private_swiftinterface` preferred, matching `transitive_swift_dependency_inputs`) enters compile action `inputs`; `.swiftmodule` moves to `unused_inputs`. `actions.run(unused_inputs_list=...)` keeps swiftmodules in the sandbox but excludes their content from the cache key.
- If a transitive dep lacks `.swiftinterface` (e.g. built without `library_evolution`), its swiftmodule falls back into `inputs` — mixed graphs are correct, only the library-evolution portion of the graph gets the caching benefit.
- Three configurators handle the split: `_dependencies_swiftmodules_configurator`, `_dependencies_swiftmodules_and_swiftdocs_configurator`, `_explicit_swift_module_map_configurator`. `_explicit_swift_module_map_info` is also feature-aware.

Defaults unchanged. Small projects and non-library-evolution users see no behavior difference.

## Non-goals

- **Not asking to change any defaults.** `library_evolution` / `emit_swiftinterface` / `use_swiftinterface_for_caching` all stay opt-in.
- **Not asking to make this the default** even for library-evolution builds — configurator branching stays behind the flag.
- Not touching the compiler command line or the file set it sees — same inputs land in the sandbox, same output shape.

## Open questions

1. Naming: `swift.use_swiftinterface_for_caching` vs a shorter alternative — happy to rename.
2. Relationship to `SWIFT_FEATURE_CACHEABLE_SWIFTMODULES`: orthogonal (that one strips embedded absolute paths from swiftmodule to make it portable; this one keys on swiftinterface entirely), but naming may be confusable.
3. Tests: happy to add a `build_test` covering (a) feature-on with library-evolution deps confirming `unused_inputs_list` emission, (b) feature-on with mixed deps confirming fallback, (c) feature-off control. Not landed in the PR yet since the shape of tests here varies; ok to add before merge.

## PR

#1880 Verified end-to-end against the production monorepo referenced above.

Contributor guide

Open the contributing guide

Research direction

Start with PR #1880 and trace the three named configurators: _dependencies_swiftmodules_configurator, _dependencies_swiftmodules_and_swiftdocs_configurator, and _explicit_swift_module_map_configurator, along with _explicit_swift_module_map_info. Review the proposed build_test cases for feature-on, mixed dependencies, and feature-off behavior; done means the opt-in behavior and fallback are covered without changing defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
swift
Domain
build-system
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.