apache / apache/openwhisk

ThrottleTests heisenbug

Open
#4,139 0 comments 0 reactions 0 assignees View on GitHub
heisenbug
Dominant language
Scala
Stars
6.8k
Forks
1.2k
Avg merge
2d 14h
Merged PRs (30d)
2

Description

Seen failure in `limits.ThrottleTest`

```
org.scalatest.exceptions.TestFailedException
:
error waiting for activation 1eb40fc18d684c39b40fc18d683c3927 for 5 minutes: Cannot find activation id from 'null'
Open stacktrace
exitCode = 173 stderr = Too many concurrent requests in flight (count: 31, allowed: 30).
exitCode = 173 stderr = Too many concurrent requests in flight (count: 33, allowed: 30).
Executed 36 requests, maximum was 36, need to retry 0 (retries left: 3)
36 slow invokes (dur = 19 sec) took 2 seconds to issue
Sleeping for 15 sec
exitCode = 173 stderr = Too many concurrent requests in flight (count: 34, allowed: 30).
exitCode = 173 stderr = Too many concurrent requests in flight (count: 34, allowed: 30).
exitCode = 173 stderr = Too many concurrent requests in flight (count: 34, allowed: 30).
exitCode = 173 stderr = Too many concurrent requests in flight (count: 34, allowed: 30).
Executed 4 requests, maximum was 4, need to retry 0 (retries left: 3)
4 fast invokes (dur = 4 sec) took 0 seconds to issue
number of throttled activations: 6 out of 40
Waiting for 44 seconds, already waited for 15 seconds
clearing activations
waiting for activations to complete
Exception occurred during test execution: org.scalatest.exceptions.TestFailedException: error waiting for activation 1eb40fc18d684c39b40fc18d683c3927 for 5 minutes: Cannot find activation id from 'null'
org.scalatest.exceptions.TestFailedException: error waiting for activation 1eb40fc18d684c39b40fc18d683c3927 for 5 minutes: Cannot find activation id from 'null'
at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:528)
at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:527)
at org.scalatest.FlatSpec.newAssertionFailedException(FlatSpec.scala:1685)
at org.scalatest.Assertions.fail(Assertions.scala:1089)
at org.scalatest.Assertions.fail$(Assertions.scala:1085)
```
https://scans.gradle.com/s/m3phkzpg3v52i/tests/xaqugzlgs2zw6-6whyek7tx2eem

Contributor guide

Open the contributing guide

Research direction

Start by reproducing limits.ThrottleTest and reviewing the linked Gradle test scan alongside its concurrency and activation-waiting output. Determine why the test can fail to find an activation under concurrent requests; done means the test completes reliably without the reported throttling or missing-activation failure.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.