linkerd / linkerd/linkerd2

Tracing test is flakey

Open
#7,538 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

area/test area/tracing help wanted
Dominant language
Go
Stars
11.5k
Forks
1.4k
Avg merge
3d 22h
Merged PRs (30d)
53

Description

The tracing test frequently fails with spurious errors like:

```
--- FAIL: TestTracing (179.94s)
--- FAIL: TestTracing/expect_full_trace (120.03s)
tracing_test.go:224: No trace found with processes: [nginx web emoji voting linkerd-proxy]
FAIL
FAIL github.com/linkerd/linkerd2/test/integration/tracing 180.065s
FAIL
```

That is, after running other tests, we wait 3 minutes for the test to fail, invalidating the whole CI run.

I'm going to disable this test until we can get at the root of this flakiness.

See also #7403

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with test/integration/tracing/tracing_test.go, especially TestTracing/expect_full_trace and the failure at line 224. Reproduce the tracing integration test after the preceding tests, identify why the full trace is intermittently missing, and verify that the test no longer waits three minutes before failing.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
observability-sre, testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.