rust-lang / rust-lang/mdBook

Flaky GUI tests with heading-nav

Open
#2,895 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

A-testing-mdbook-iteself
Dominant language
Rust
Stars
22.2k
Forks
1.9k
PR merge metrics
PR metrics pending

Description

The following errors happened on CI:

..FF..........                                    (15/15)

======== tests/gui/heading-nav-markup.goml ========

Error:  `tests/gui/heading-nav-markup.goml` line 8: The following errors happened: [`Heading with html` isn't equal to `Heading with code or italic or bold or strike`]: for command `assert-text: (".current-header", "Heading with code or italic or bold or strike")`
Error:  `tests/gui/heading-nav-markup.goml` line 9: The following errors happened (for selector `.current-header`): [expected `Heading with <code>code</code> or <em>italic</em> or <strong>bold</strong> or <del>strike</del>` for property `"innerHTML"`, found `Heading with <span>html</span>`]: for command `assert-property: (".current-header", {"innerHTML": "Heading with <code>code</code> or <em>italic</em> or <strong>bold</strong> or <del>strike</del>"})`

======== tests/gui/heading-nav-normal-intro.goml ========

Error:  `tests/gui/heading-nav-normal-intro.goml` line 8: The following errors happened: [`And a sub heading` isn't equal to `The first heading`]: for command `assert-text: (".current-header", "The first heading")`


<= doc-ui tests done: 13 succeeded, 2 failed, 0 filtered out

There's likely some race where the test isn't properly waiting or something, I'm not sure.

Happened in https://github.com/rust-lang/mdBook/pull/2891

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running tests/gui/heading-nav-markup.goml and tests/gui/heading-nav-normal-intro.goml, focusing on the assertions at lines 8–9 and the reported heading-navigation failures. Reproduce the CI behavior and inspect whether those assertions wait for navigation and rendered markup to settle. Done means both tests pass reliably, including their text and innerHTML assertions.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.