Comcast / Comcast/sirius

AskTimeoutExceptions during startup

Open
#131 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
296
Forks
48
PR merge metrics
No merged PRs in 30d

Description

We sometimes see `akka.pattern.AskTimeoutException` during startup, and generally a restart fixes the problem or the timeouts will occur several times and then cease.
The timeouts occur while executing:

`sirius.enqueuePut(event.getKey().fullKey(), data).get();`

As you can see, we enqueuePut, and then immediately block until the future returns. Here is a [thread dump](https://github.com/Comcast/sirius/files/1162716/blah3.txt) I took while waiting for the future.

The main thread is waiting for the future, but there don't seem to be any other threads doing anything sirius related. I had expected to see our main thread waiting on a sirius implementation thread to finish its work, but I don't see any threads like that.

This node's sirius cluster config only contained `/user/sirius`.

This is happening at startup, but after `com.comcast.xfinity.sirius.api.Sirius.isOnline()` returns true. Sometime around then, we see this in the logs:

> 2017-07-14 19:46:24,297 WARN s=localListingInfoWebService-root_out env="ape" [sirius-system-akka.actor.default-dispatcher-12] Sirius(akka://sirius-system): SiriusSupervisor Actor received unrecognized message IsInitializedResponse(true)

Perhaps this warning is relevant to the issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.