cucumber / cucumber/cucumber-ruby

Upgrading from 11.0.0 to 11.1.1 causes a big increase in time for message creation

Open
#1,902 7 comments 1 reaction 0 assignees View on GitHub
:bank: debt envelope-migration
Dominant language
Ruby
Stars
5.2k
Forks
1.1k
Avg merge
1d 16h
Merged PRs (30d)
3

Description

### 👓 What did you see?

My tests consist of 1584 scenarios and 18147 steps. On 11.0.0 this takes roughly 3 mins. Since the bump, it takes at least 4.5mins.

AI did a bit of digging and profiling, and it pinned it down to the following change in the MessageBuilder. I've included it, it might be a helpful pointer but obvious you folks know best what the reason for that change was.

---

Cucumber 11.1.x rewrote Formatter::MessageBuilder#on_test_step_started / #on_test_step_finished. What was an O(1) hash lookup:

```
test_case = @test_case_by_step_id[event.test_step.id]
```

became a full scan of every test case accumulated so far, checking every step of each:

```
@repository.test_case_by_id.values
.detect { |tc| tc.test_steps.any? { |s| s.id == event.test_step.id } }
```

That runs twice per step, and the repository grows as the run progresses, so the cost is quadratic in suite size.

### ✅ What did you expect to see?

Upgrading should not impact the test run in such a big way.

### 📦 Which tool/library version are you using?

_No response_

### 🔬 How could we reproduce it?

_No response_

### 📚 Any additional context?

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with Formatter::MessageBuilder#on_test_step_started and #on_test_step_finished, comparing their 11.0.0 and 11.1.x behavior and the repository data they use. Reproduce or profile a large suite like the reported 1,584 scenarios and 18,147 steps; done means message creation no longer introduces the reported quadratic slowdown while existing formatter behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
ruby
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.