webdriverio / webdriverio/webdriverio
Cache list of spec files
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 9.8k
- Forks
- 2.7k
- Avg merge
- 2d 13h
- Merged PRs (30d)
- 69
Description
Is your feature request related to a problem? Please describe.
Finding specs in a big monorepo takes ages. Think 130K+ files (sources + tests + assets).
Rough directory structure is this:
page1/
- features/
page1.feature
- subpage1/
- features/
- subpage1.feature
- subpage1.js
- page1.js
- page1.css
page2/
<same structure>
ui/shared_component/
- features/
- shared_component_in_page1.feature
- shared_component_in_page2.feature
- shared_component.js
This is a screenshot from a JS CPU profile of the first ~2 minutes of running WDIO on our repo. At that point, it didn't even get to actually running the tests.
Note how consistently ConfigParser#getSpecs appears in the chart, and how much time it consumes.
This massive time is to find just ~250 .feature files among the sources.
I also had passed a tag expression that would filter that down to just 4 files, so it's unacceptable that WDIO takes this long.
Describe the solution you'd like
ConfigParser#getSpecs could be memoized, so that when it takes the same input, it always returns the same value.
I guess that adding/removing files could become a problem with that solution, in which case it should be coupled with a file system watching solution to manipulate the cache.
Describe alternatives you've considered
The specs option was too inclusive (src/**/features/*.feature), so I tinkered with the suites option:
suites: {
page1: ['src/page1/**/features/*.feature', 'src/ui/**/features/*.feature']
// and so on
}
This didn't help bringing the time to boot up down noticeably, though.
And, as probably obvious from above, keeping a list of .feature files in the config that doesn't make use of wildcards is impossible.
Additional context
Node.js v10.16.3
@wdio packages used:
"@wdio/cli": "^5.16.10",
"@wdio/cucumber-framework": "^5.16.10",
"@wdio/local-runner": "^5.16.10",
"@wdio/selenium-standalone-service": "^5.16.10",
"@wdio/spec-reporter": "^5.16.10",
"@wdio/sync": "^5.16.10",
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with ConfigParser#getSpecs in packages/wdio-config/src/lib/ConfigParser.js and review how repeated spec discovery is triggered for large repositories. Define how cached results should respond to added or removed files, then verify that broad glob patterns and tag-filtered feature specs no longer incur the same repeated discovery cost.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js
- Domain
- performance, testing-qa, tooling
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100