rubyforgood / rubyforgood/casa

Speed up the RSpec suite: parallelize CI, then shrink the JS system-spec slice

Open
#7,173 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

📈 Performance Improvement 🧪 Tests
Dominant language
Ruby
Stars
381
Forks
542
Avg merge
3d 15h
Merged PRs (30d)
47

Description

Baseline (measured, not guessed)

The CI rspec step on main takes 8.4 min, in one process, with no parallelism (bundle exec rspec).

Local timings by slice:

Slice Examples Time Per example
spec/system :js 368 498s 1.35s
spec/requests 932 50s 0.05s
spec/system (rack_test) 388 36s 0.09s
spec/models 666 22s 0.03s

JS system specs are ~82% of the wall clock for ~16% of the examples, and they ran at 28% CPU — the process sits idle waiting on Chrome. That single fact sets the priority order: parallelize first, then shrink the JS set.

Inside the JS slice:

  • spec/system/accessibility/axe_spec.rb — 45 examples / 118s (24% of JS time)
  • spec/system/casa_cases/edit_spec.rb — 26 examples / 61s
  • spec/system/typeahead_controls_spec.rb1 example / 30s
  • spec/system/layouts/flashes_spec.rb — 4 examples / 18.5s

Two things checked and ruled out as meaningful levers: Prosopite's per-example scan costs only ~4% (measured A/B on spec/models, 28.4s vs 29.7s), and Build App + assets:precompile is 0.2 min. Neither is worth touching early.

Phase 1 — turn on parallelism (config only, no spec edits)

The repo is already fully wired for parallel_tests and just doesn't use it in CI. The gem is in the Gemfile, and every per-process collision point is already parameterized by TEST_ENV_NUMBER:

  • config/database.yml:17 — per-process database name
  • config/storage.yml:3 — per-process Active Storage root
  • spec/rails_helper.rb:132 — per-process Capybara server port
  • spec/support/download_helpers.rb:3 — per-process download dir
  • spec/spec_helper.rb:18 — per-process SimpleCov command_name

Someone built this and it fell out of CI. Tasks:

  • In .github/workflows/rspec.yml, replace bundle exec rake db:create db:schema:load with bundle exec rake parallel:create parallel:load_schema, and bundle exec rspec with bundle exec parallel_test spec --type rspec -n 4.
  • Cache tmp/parallel_runtime_rspec.log across runs so --group-by runtime balances groups. Without it, whichever group holds axe_spec becomes the critical path.
  • Add a SimpleCov merge step — bump SimpleCov.merge_timeout and collate the per-process resultsets before the qlty upload, or the reported coverage will silently drop.

ubuntu-latest on a public repo is 4 vCPU. Because the JS specs are browser-wait-bound, 4 processes should scale close to linearly: ~8.4 min → ~3 min. Memory headroom needs a check — 4 concurrent headless Chromes in 16 GB should be fine, but watch it.

Phase 2 — shrink the JS slice

  • Audit the :js tags. 368 JS examples is far more than a Hotwire app of this size needs. Every one converted to rack_test goes 1.35s → 0.09s (15×). Highest-yield item on the list, but it's per-file judgment work, so no payoff number until the audit is done.
  • Move spec/system/accessibility/axe_spec.rb into its own CI job. 118s of full page-load + axe-core runs, independent of everything else. Running it concurrently takes ~2 min off the critical path at zero risk.
  • Fix the flakes, then cut retry: 3. spec/support/rspec_retry.rb retries every :js example up to 3× in CI, so the slowest specs are also the ones that can triple in cost. spec/system/learning_hours/date_range_spec.rb failed 3 examples in a local run and is burning retries in CI right now. Fix it, then drop to retry: 1.
  • Fix spec/system/typeahead_controls_spec.rb. One example, 30s, ~20 let! records, two explicit sleeps and a Timeout.timeout(6) poll loop. Split it per control and replace the manual polling with Capybara's own waiting.

Phase 3 — keep it from regressing

  • Add config.profile_examples = 10 behind an env var so the next slow spec is visible without a manual profiling run.
  • Set .prosopite_ignore directories to :off rather than :log_only. spec/system is on that list, so its N+1s can never fail the build, yet the slowest 82% of the suite still pays the scan cost. Only worth ~4% though — do it last.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with .github/workflows/rspec.yml and compare its current commands with the per-process settings in config/database.yml, config/storage.yml, spec/rails_helper.rb, spec/support/download_helpers.rb, and spec/spec_helper.rb. Run the documented baseline and inspect the SimpleCov and runtime-log handling before choosing a phase. Done means the selected parallelism, balancing, coverage, or spec-speed change works without regressing CI results.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, rails, ruby
Domain
ci-cd, performance, testing-qa
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.