envoyproxy / envoyproxy/gateway
Flaky E2E Test: UDPRouteBackendFQDNTest/UDPRoute_with_a_FQDN_type_Backend
- Dominant language
- Go
- Stars
- 3k
- Forks
- 864
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 140
Description
## Problem
The E2E test `TestE2E/UDPRouteBackendFQDNTest/UDPRoute_with_a_FQDN_type_Backend` is flaky and fails intermittently with UDP DNS query timeouts.
## Evidence
### Recent Failures
1. **PR #9359**: `e2e-test (v1.34.8, ipv4, default)` - FAIL
- Error: `failed to perform a UDP query: read udp 172.18.0.1:58505->172.18.0.206:5300: i/o timeout`
- Run: https://github.com/envoyproxy/gateway/actions/runs/28210980762/job/83573440003
2. The same test passes in other configurations:
- v1.33.12 ipv4 - PASS
- v1.35.5 ipv6 - PASS
- v1.36.1 all configs - PASS
3. On main branch recent runs, `e2e-test (v1.34.8, ipv4, default)` passes:
- Run: https://github.com/envoyproxy/gateway/actions/runs/28210631971
### Pattern
- Fails only on **Kubernetes v1.34.8 + IPv4** configuration
- Failure is always **UDP DNS timeout** (not a code issue)
- Passes on other K8s versions and IPv6
- Passes when rerun
## Root Cause
This appears to be an infrastructure/CI environment issue:
1. DNS resolution timeout in UDP queries
2. Specific to K8s v1.34.8 cluster setup
3. Network/DNS configuration issue in the test environment
## Impact
This flaky test blocks legitimate PRs from passing CI, requiring manual retest or approval.
## Suggested Fixes
1. **Immediate**: Increase UDP timeout in the test (currently appears to be 2s, could increase to 5s)
2. **Investigation**: Check DNS/CoreDNS configuration in v1.34.8 cluster setup
3. **Long-term**: Consider making the test more resilient to network delays
## Test Location
- File: `test/e2e/tests/udproute_with_backend.go:96`
- Test: `UDPRouteBackendFQDNTest`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with test/e2e/tests/udproute_with_backend.go:96 and the UDP query timeout used by UDPRouteBackendFQDNTest. Reproduce the Kubernetes v1.34.8 IPv4/default E2E configuration and compare its DNS or CoreDNS behavior with passing configurations. Done means the test no longer flakes in that configuration without masking genuine failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- ci-cd, networking, testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100