apache / apache/pulsar

[Bug] Caused by: org.apache.pulsar.common.util.FutureUtil$LowOverheadTimeoutException: Failed to load topic within timeout

Open
#18,162 3 comments 0 reactions 0 assignees View on GitHub
Stale type/bug
Dominant language
Java
Stars
15.3k
Forks
3.8k
Avg merge
1d 14h
Merged PRs (30d)
160

Description

### Search before asking

- [ ] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.

> 4135 Oct 23 21:33:20 test_host pulsar[123890]: 2022-10-23T21:33:20,366+0800 [pulsar-io-4-79] ERROR org.apache.pulsar.broker.service.ServerCnx - [/10.225.80.45:58017] Fa
4135 iled to create topic persistent://benchmark/ns-test/test-test-reusing-topic500-0000-partition-417, producerId=835
4136 Oct 23 21:33:20 test_host pulsar[123890]: java.util.concurrent.CompletionException: org.apache.pulsar.common.util.FutureUtil$LowOverheadTimeoutException: Failed to
4136 load topic within timeout
4137 Oct 23 21:33:20 test_host pulsar[123890]: at java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:332) ~[?:?]
4138 Oct 23 21:33:20 test_host pulsar[123890]: at java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:347) ~[?:?]
4139 Oct 23 21:33:20 test_host pulsar[123890]: at java.util.concurrent.CompletableFuture$UniCompose.tryFire(CompletableFuture.java:1141) ~[?:?]
4140 Oct 23 21:33:20 test_host pulsar[123890]: at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:510) ~[?:?]
4141 Oct 23 21:33:20 test_host pulsar[123890]: at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:2162) ~[?:?]
4142 Oct 23 21:33:20 test_host pulsar[123890]: at org.apache.pulsar.common.util.FutureUtil.lambda$addTimeoutHandling$9(FutureUtil.java:198) ~[org.apache.pulsar-pulsar-c
4142 ommon-2.11.0.jar:2.11.0]
4143 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.PromiseTask.runTask(PromiseTask.java:98) ~[io.netty-netty-common-4.1.77.Final.jar:4.1.77.Fina
4143 l]
4144 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.ScheduledFutureTask.run(ScheduledFutureTask.java:170) ~[io.netty-netty-common-4.1.77.Final.ja
4144 r:4.1.77.Final]
4145 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174) ~[io.netty-netty-common-4.1.77.
4145 Final.jar:4.1.77.Final]
4146 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167) ~[io.netty-netty-common-4.1
4146 .77.Final.jar:4.1.77.Final]
4147 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470) ~[io.netty-netty-co
4147 mmon-4.1.77.Final.jar:4.1.77.Final]
4148 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:391) ~[io.netty-netty-transport-classes-epoll-4.1.77.Fin
4148 al.jar:4.1.77.Final]
4149 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) ~[io.netty-netty-common-4
4149 .1.77.Final.jar:4.1.77.Final]
4150 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[io.netty-netty-common-4.1.77.Final.jar:4.1
4150 .77.Final]
4151 Oct 23 21:33:20 test_host pulsar[123890]: at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) ~[io.netty-netty-common-4.1.77.F
4151 inal.jar:4.1.77.Final]
4152 Oct 23 21:33:20 test_host pulsar[123890]: at java.lang.Thread.run(Thread.java:833) ~[?:?]
4153 Oct 23 21:33:20 test_host pulsar[123890]: Caused by: org.apache.pulsar.common.util.FutureUtil$LowOverheadTimeoutException: Failed to load topic within timeout
4154 Oct 23 21:33:20 test_host pulsar[123890]: at org.apache.pulsar.broker.service.BrokerService.futureWithDeadline(...)(Unknown Source) ~[org.apache.pulsar-pulsar-brok
4154 er-2.11.0.jar:2.11.0]

### Version

Pulsar: 2.11.0

### Minimal reproduce step
I don't know how to reproduce this issue. It happens several times in my openmessging-benchmark test

### What did you expect to see?

Load topic success

### What did you see instead?

Caused by: org.apache.pulsar.common.util.FutureUtil$LowOverheadTimeoutException: Failed to load topic within timeout

### Anything else?

_No response_

### Are you willing to submit a PR?

- [X] I'm willing to submit a PR!

Contributor guide

Open the contributing guide

Research direction

Start with the failure path in org.apache.pulsar.broker.service.ServerCnx and BrokerService.futureWithDeadline, then inspect FutureUtil.java around line 198 where the timeout is added. Use the provided Pulsar 2.11.0 log and openmessaging-benchmark context to determine why loading the topic exceeds the deadline. Done means identifying a reproducible cause and defining a verified fix or regression coverage.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.