cucumber / cucumber/cucumber-ruby
Upgrading from 11.0.0 to 11.1.1 causes a big increase in time for message creation
- Dominant language
- Ruby
- Stars
- 5.2k
- Forks
- 1.1k
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 3
Description
### 👓 What did you see?
My tests consist of 1584 scenarios and 18147 steps. On 11.0.0 this takes roughly 3 mins. Since the bump, it takes at least 4.5mins.
AI did a bit of digging and profiling, and it pinned it down to the following change in the MessageBuilder. I've included it, it might be a helpful pointer but obvious you folks know best what the reason for that change was.
---
Cucumber 11.1.x rewrote Formatter::MessageBuilder#on_test_step_started / #on_test_step_finished. What was an O(1) hash lookup:
```
test_case = @test_case_by_step_id[event.test_step.id]
```
became a full scan of every test case accumulated so far, checking every step of each:
```
@repository.test_case_by_id.values
.detect { |tc| tc.test_steps.any? { |s| s.id == event.test_step.id } }
```
That runs twice per step, and the repository grows as the run progresses, so the cost is quadratic in suite size.
### ✅ What did you expect to see?
Upgrading should not impact the test run in such a big way.
### 📦 Which tool/library version are you using?
_No response_
### 🔬 How could we reproduce it?
_No response_
### 📚 Any additional context?
_No response_
Contributor guide
Research direction
Start with Formatter::MessageBuilder#on_test_step_started and #on_test_step_finished, comparing their 11.0.0 and 11.1.x behavior and the repository data they use. Reproduce or profile a large suite like the reported 1,584 scenarios and 18,147 steps; done means message creation no longer introduces the reported quadratic slowdown while existing formatter behavior remains intact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- ruby
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100