kmesh-net / kmesh-net/kmesh

TestKmeshRestart flake

Open
#698 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
757
Forks
276
Avg merge
5d 20h
Merged PRs (30d)
2

Description

https://github.com/kmesh-net/kmesh/actions/runs/10276792570/job/28437778043?pr=695
```
=== RUN TestKmeshRestart
2024-08-07T02:03:57.129439Z info tf === BEGIN: Test: '_home_runner_work_kmesh_kmesh_test_e2e[TestKmeshRestart]' ===
2024-08-07T02:03:57.129815Z info port forward completed without error
2024-08-07T02:04:03.149133Z info tf Checking pods ready...
2024-08-07T02:04:03.151076Z info tf [ 0] kmesh-9nlrq Running (Ready)
2024-08-07T02:04:03.151091Z info tf [ 1] kmesh-kf9kn Running (Ready)
result.go:66: Minimum success threshold, 1.000000, was not met. 194/195 (0.994872) requests failed: 1 error occurred:
* request 3: failed calling enrolled-to-kmesh (cluster=cluster-0)->'[http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local:80](http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local/)': call failed from enrolled-to-kmesh (cluster=cluster-0) to [http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local:80](http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local/) (using http): response[0]: expected response code `200`, got "503". Response: RawContent: [0] Url=[http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local:80](http://service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local/)
[0] SourceIP=10.244.1.7
[0] Latency=2.91493ms
[0] ActiveRequests=1
[0] StatusCode=503
[0] ResponseHeader=Content-Length:0
[0] ResponseHeader=Date:Wed, 07 Aug 2024 02:03:56 GMT
[0] ResponseHeader=Server:envoy
[0] ResponseHeader=X-Envoy-Decorator-Operation:service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local:80/*
[0] ResponseHeader=X-Envoy-Upstream-Service-Time:0

ID:
Method:
Protocol:
Alpn:
URL:
Version:
Port:
Code: 503
Host:
Hostname:
Cluster:
IstioVersion:
IP: 10.244.1.7
Request Headers: map[]
Response Headers: map[Content-Length:[0] Date:[Wed, 07 Aug 2024 02:03:56 GMT] Server:[envoy] X-Envoy-Decorator-Operation:[service-with-waypoint-at-service-granularity.echo-1-48601.svc.cluster.local:80/*] X-Envoy-Upstream-Service-Time:[0]]


2024-08-07T02:04:03.151192Z info tf === DONE (failed): Test: '_home_runner_work_kmesh_kmesh_test_e2e[TestKmeshRestart] (6.021750741s)' ===
--- FAIL: TestKmeshRestart (6.02s)
FAIL
2024-08-07T02:04:03.151237Z info tf === FAILED: Test Run: '_home_runner_work_kmesh_kmesh_test_e2e' (exitCode: 1) ===
2024-08-07T02:04:03.151244Z info tf === Suite "_home_runner_work_kmesh_kmesh_test_e2e" run time: 38.851905389s ===
2024-08-07T02:04:03.151253Z info tf Wrote trace to /tmp/trace.yaml
2024-08-07T02:04:03.152065Z info port forward completed without error
2024-08-07T02:04:03.152315Z info port forward completed without error
2024-08-07T02:04:03.152502Z info port forward completed without error
2024-08-07T02:04:03.152605Z info port forward completed without error
2024-08-07T02:04:03.153762Z info port forward completed without error
FAIL kmesh.net/kmesh/test/e2e 38.903s
FAIL
make: *** [Makefile:151: e2e] Error 1
Error: Process completed with exit code 2.
```

Contributor guide

Open the contributing guide

Research direction

Start in test/e2e with TestKmeshRestart and reproduce the failure using the e2e target in Makefile:151. Review the linked GitHub Actions run and the emitted trace to determine why one request returns Envoy 503 after the pods report ready. Done means the restart test reliably meets its 100% success threshold.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
distributed-systems, networking, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.