Flaky GUI tests with heading-nav
Open
Nobody has claimed this yet.
A-testing-mdbook-iteself
- Dominant language
- Rust
- Stars
- 22.2k
- Forks
- 1.9k
- PR merge metrics
- PR metrics pending
Description
The following errors happened on CI:
..FF.......... (15/15)
======== tests/gui/heading-nav-markup.goml ========
Error: `tests/gui/heading-nav-markup.goml` line 8: The following errors happened: [`Heading with html` isn't equal to `Heading with code or italic or bold or strike`]: for command `assert-text: (".current-header", "Heading with code or italic or bold or strike")`
Error: `tests/gui/heading-nav-markup.goml` line 9: The following errors happened (for selector `.current-header`): [expected `Heading with <code>code</code> or <em>italic</em> or <strong>bold</strong> or <del>strike</del>` for property `"innerHTML"`, found `Heading with <span>html</span>`]: for command `assert-property: (".current-header", {"innerHTML": "Heading with <code>code</code> or <em>italic</em> or <strong>bold</strong> or <del>strike</del>"})`
======== tests/gui/heading-nav-normal-intro.goml ========
Error: `tests/gui/heading-nav-normal-intro.goml` line 8: The following errors happened: [`And a sub heading` isn't equal to `The first heading`]: for command `assert-text: (".current-header", "The first heading")`
<= doc-ui tests done: 13 succeeded, 2 failed, 0 filtered out
There's likely some race where the test isn't properly waiting or something, I'm not sure.
Happened in https://github.com/rust-lang/mdBook/pull/2891
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running tests/gui/heading-nav-markup.goml and tests/gui/heading-nav-normal-intro.goml, focusing on the assertions at lines 8–9 and the reported heading-navigation failures. Reproduce the CI behavior and inspect whether those assertions wait for navigation and rendered markup to settle. Done means both tests pass reliably, including their text and innerHTML assertions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100