Flaky tests: Asynchronous queries can return unmounted elements

Edit: Skip comments about debugging and jump to the root cause.

• If your issue is regarding one of the query APIs (getByText,
  getByLabelText, etc), then please file it on the dom-testing-library
  repository instead. If you file it here it will be closed. Thanks :)

This is related to asynchronous queries (findBy*) but the issue is not on dom-testing-library. See https://github.com/testing-library/dom-testing-library/issues/876.

• @testing-library/react version: "^11.1.0"
• Testing Framework and version:
  • fresh project generated with CRA 4.0.1
  • jest@26.6.0
• DOM Environment:
  • jest-environment-jsdom@26.6.2
  • jsdom@16.4.0

Relevant code or config:

Also available in codesandbox below.

This issue comes up randomly so below we are generating 200 tests:

import React, { useEffect, useState } from "react";
import { render, waitForElementToBeRemoved, screen } from "@testing-library/react";

function Component() {
    const [visible, setVisible] = useState(false);

    useEffect(() => {
        const timer1 = setTimeout(() => setVisible(true), 10);
        const timer2 = setTimeout(() => setVisible(false), 15);

        return () => {
            clearTimeout(timer1);
            clearTimeout(timer2);
        };
    }, []);

    return (
        <div>
            {visible && <span id="test-id">Content</span>}
        </div>
    );
}

for(const i of Array(200).fill(null).map((_, i) => i)) {
    test(`unstable test ${i}`, async () => {
        render(<Component />);

        const content = await screen.findByText("Content");
        await waitForElementToBeRemoved(content);
    });
}

Above is only a minimal repro. My real use case includes querying "Loading" text which is visible while MSW is handling the request. It does not use setTimeout at all. Something like:

// Click submit button
userEvent.click(screen.getByRole("button", { name: "Submit" }));

// Loader should appear
const loader = await screen.findByText("Loading");

// Loader should disappear once request resolves
await waitForElementToBeRemoved(loader);

What you did:

Queried element with asynchronous query findBy*. I would expect element returned to be present in the DOM. If it wasn't the query should throw error.
Passed the queried element to waitForElementToBeRemoved.

What happened:

Element returned from findByText was not in DOM when waitForElementToBeRemoved started handling it.

 FAIL  src/App.test.js
  × unstable test (66 ms)

  ● unstable test

    The element(s) given to waitForElementToBeRemoved are already removed. waitForElementToBeRemoved requires that the element(s) exist(s) before waiting for removal.

      33 |     render(<Component />);
      34 |     const content = await screen.findByText("Content");
    > 35 |     await waitForElementToBeRemoved(content);
         |           ^
      36 | });
      37 |
      38 |

      at initialCheck (node_modules/@testing-library/dom/dist/wait-for-element-to-be-removed.js:16:11)      
      at waitForElementToBeRemoved (node_modules/@testing-library/dom/dist/wait-for-element-to-be-removed.js:39:3)
      at Object.<anonymous> (src/App.test.js:35:11)

Most of the time these issues come up really randomly. We might have successful builds for months on CI and then this error occurs. Triggering new build usually fixes this. 😄

Reproduction:

The codesandbox is still showing incorrect error message Cannot read property 'parentElement' of null but this was fixed in https://github.com/testing-library/dom-testing-library/issues/871.

• React: https://codesandbox.io/s/dom-testing-libraryissues876-2-fe3fi?file=/src/React.test.js

• DOM: https://codesandbox.io/s/dom-testing-libraryissues876-2-fe3fi?file=/src/Dom.test.js

Problem description:

Asynchronous queries can return elements which have unmounted the DOM.

I was hoping the initialCheck done by waitForElementToBeRemoved would have been the cause but it looks like the parentElement is not even present before element is given to it.

const content = await screen.findByText("Content");
console.log(content.outerHTML); // <span id="test-id">Content</span>
console.log(content.parentElement); // null
await waitForElementToBeRemoved(content); // The element(s) given to waitForElementToBeRemoved are already removed.

By setting the isAsyncActSupported to false from dist/act-compat.js this issue disappears. However errors fill stdout but tests are passing. I have no idea what I'm doing here but this works 😃 .

https://github.com/testing-library/react-testing-library/blob/deafd51bc20209612674dbdaba920750c1d6352e/src/act-compat.js#L20

I think act-compact.js is causing asynchronous queries to run additional cycles/ticks/micro-queues instead of returning the element instantly. This causes setTimeouts and/or Promises to run before the element is returned. In practice the queried element can unmount before asynchronous query is resolved.

Suggested solution:

I'm not familiar with the act wrapper of RTL but I think the fix should be applied there.

There is a workaround for this: Don't use waitForElementToBeRemoved.

const content = await screen.findByText("Content");
await waitFor(() => expect(content).not.toBeInTheDocument());

Start with the supplied 200-test reproduction and inspect src/act-compat.js, especially the isAsyncActSupported path; compare it with the reported behavior when that support is disabled. Confirm the failure involves an element unmounting before waitForElementToBeRemoved begins, then add or update a regression test showing asynchronous queries do not return unmounted elements.