apache / apache/beam

[Failing Test]: Flake in apache_beam.runners.worker.statecache_test.StateCacheTest.test_concurrent_get_waits

Open
#25,433 0 comments 0 reactions 0 assignees View on GitHub
bug flake P2 python tests
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

Came across this error in one of precommits. Seems like a flake but don't recall seeing it before.

cc: @lukecwik who may be familiar with the test scenario FYI.

```Error Message
AssertionError: 1675794730365251754 not less than or equal to 1675794730365097126
Stacktrace
self =

def test_concurrent_get_waits(self):
event = threading.Semaphore(0)
threads_running = threading.Barrier(3)

def wait_for_event(key):
with cache._lock:
self.assertTrue(isinstance(cache._cache["key"], _LoadingValue))
event.release()
return "value"

cache = StateCache(5 << 20)

def load_key(output):
threads_running.wait()
output["value"] = cache.get("key", wait_for_event)
output["time"] = time.time_ns()

t1_output = {}
t1 = threading.Thread(target=load_key, args=(t1_output, ))
t1.start()

t2_output = {}
t2 = threading.Thread(target=load_key, args=(t2_output, ))
t2.start()

# Wait for both threads to start
threads_running.wait()
# Record the time and wait for the load to start
current_time_ns = time.time_ns()
event.acquire()
t1.join()
t2.join()

# Ensure that only one thread did the loading and not both by checking that
# the semaphore was only released once
self.assertFalse(event.acquire(blocking=False))

# Ensure that the load time is greater than the set time ensuring that
# both loads had to wait for the event
> self.assertLessEqual(current_time_ns, t1_output["time"])
E AssertionError: 1675794730365251754 not less than or equal to 1675794730365097126

apache_beam/runners/worker/statecache_test.py:293: AssertionError
```

### Issue Failure

Failure: Test is flaky

### Issue Priority

Priority: 2 (backlog / disabled test but we think the product is healthy)

### Issue Components

- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.