LivelyKernel / LivelyKernel/lively.next
Some `lively.morphic/text` tests regularly fail in CI
@linusha is already working on this.
Since May 16, 2024.
- Dominant language
- JavaScript
- Stars
- 90
- Forks
- 20
- PR merge metrics
- No merged PRs in 30d
Description
Ever since merging #1481 our CI pipelines failed more or less reliably.
We often had trouble reproducing this locally. Although it was not impossible to reproduce it locally, it did work more often than not and inside of a running lively environment the tests never failed.
We found out that disabling the caching inside of the actions, thus enforcing a clean reinstallation of lively reliably seems to solve the issue.
We then tried to move the lively.morphic tests, which are the failing tests, to the start of the pipeline for easier testing. This lead to the tests failing again.
We briefly experimented with setting headless: 'shell' instead of headless: 'new'. However, this would need to be investigated further.
Overall the situation is highly unsatisfactory and we have no real indicator what the exact cause for the problems is. For now, we just hope that the problem does not persist, but we should investigate how we can harden the pipeline. Enabling the caching again would also be quite nice as the runtime of the pipelines skyrockets drastically, which is especially annoying for the PR checks.
In the worst case we could try to force the installation only for the daily checks and for lively.morpic.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.