bahmutov / bahmutov/cypress-split
SPLIT_TIME_THRESHOLD is not respected when using SPLIT_OUTPUT_FILE
- Dominant language
- JavaScript
- Stars
- 285
- Forks
- 28
- PR merge metrics
- No merged PRs in 30d
Description
Wanted to open a discussion about the intended behaviour of `SPLIT_TIME_THRESHOLD` in conjunction with `SPLIT_OUTPUT_FILE`.
**Context**
In my CI setup, the original `timings.json` file is kept in `cypress/e2e//timings.json`, and I am setting the `SPLIT_TIME_THRESHOLD` to be `0.2`.
I am also using `SPLIT_OUTPUT_FILE` to save the updated timings for each parallel node into their own artifacts directory like so:
`artifacts//split-timings/node-<1,2,3,etc>/timings.json`. These are then merged back together by a later merge job using the `cypress-split-merge` binary (which is working great btw!)
**Expected Behaviour (in my head)**
The behaviour I was expecting here was that the `SPLIT_OUTPUT_FILE` would not include updated timings unless the threshold is met. If the threshold was not met, it would either:
1. not write out an updated timings.json at all (this might not work well when merging later)
2. or it would write to the `node-X` output file, but it would be the original times of the test cases and not the updated ones. in this case, when merging the updated timings, Git can realise there is no change to the merged timings file and not bother committing.
**Actual Behaviour**
Despite the changes not meeting the threshold, it writes the updated timings to the `SPLIT_OUTPUT_FILE` anyway:
I can see in the code it's because `OUTPUT` !== the `INPUT` file, so we write it anyway: https://github.com/bahmutov/cypress-split/blob/16f7a78c5183c5ccf15cdecab29b0888d368348b/src/index.js#L256
As a result of the above, later in our merge job the spec timings are being updated no matter what the threshold is.
**Question for you**
What would be the ideal way for `SPLIT_OUTPUT_FILE` to respect `SPLIT_TIME_THRESHOLD`? Would love to help hop on a solution!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at src/index.js around line 256 and trace how SPLIT_TIME_THRESHOLD, INPUT, and SPLIT_OUTPUT_FILE affect timing-file writes. Review the cypress-split-merge behavior described in the issue, then determine and test how unchanged timings should be represented in per-node output so later merging does not update timings when the threshold is unmet.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cypress, javascript
- Domain
- ci-cd, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100