apache / apache/pulsar-client-node

`AlreadyClosed` exception when using `topicsPattern`

オープン
#132 コメント 4 件 リアクション 0 件 担当者 0 名 GitHub で見る
主要言語
C++
スター
164
フォーク
98
PR マージ指標
30日以内にマージされた PR はありません

説明

We are unable to cleanly close our consumer when using `topicsPattern`:

```
2020-10-27 11:15:43.304 INFO [0x70000d6d5000] ConnectionPool:85 | Created connection for pulsar://pulsar.data-access-platform-portal.docker:6650
2020-10-27 11:15:43.307 INFO [0x70000ef61000] ClientConnection:343 | [192.168.99.1:58802 -> 192.168.99.100:6650] Connected to broker
Closing consumer...
(node:35996) UnhandledPromiseRejectionWarning: Error: Failed to close consumer: AlreadyClosed
(node:35996) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:35996) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
```

That's the result from running this code snippet (with the appropriate docker container running):

```
const Bluebird = require('bluebird')
const Pulsar = require('pulsar-client')

const doit = async () => {
const client = new Pulsar.Client({
webServiceUrl: 'http://pulsar.data-access-platform-portal.docker:8080',
serviceUrl: 'pulsar://pulsar.data-access-platform-portal.docker:6650'
})
const consumer = await client.subscribe({
topicsPattern: `non-persistent://public/default/topic`,
subscription: 'test-subscription',
subscriptionType: 'Shared',
subscriptionInitialPosition: 'Earliest'
})

// Rather than push messages to consume, we're just waiting for a second
await Bluebird.delay(1000)

console.log('Closing consumer...')
// We've also tried omitting this line, but it still produces errors
await consumer.close()
console.log('Closing client...')
await client.close()
}

doit()
```

We've tried this with node versions 10.16.3, 12.16.0, and 14.11.0 and pulsar 2.6.1.

I _suspect_ that the root cause is that it _appears_ that the consumer implementation in this library makes the assumption that there's only a single consumer - https://github.com/apache/pulsar-client-node/blob/f010fb2ed3c530af8006ba3d3a7b73c6b46f6507/src/Consumer.cc#L178 overwrites the wrapper's consumer with each invocation. While this wouldn't be a problem with a single topic, it seems that the last consumer "wins" in that it gets stitched up to be the consumer saved in the wrapper. However, there may be a race condition for whether the listener handler stitched up correctly from the config (https://github.com/apache/pulsar-client-node/blob/f010fb2ed3c530af8006ba3d3a7b73c6b46f6507/src/Consumer.cc#L184 but then the consumer config is subsequently deleted).

Looking at `ClientImpl` in the pulsar C++ code, it appears that it also takes responsibility for closing any open consumers and producers, so we tried also just relying on the client in order to close the consumers (since `ClientImpl` appears to correctly track all newly-created consumers), but that _also_ reproduces with `AlreadyClosed` getting thrown.

All that said, it is a little tricky to track all of it because of the number of levels of indirection - from the JS to the C++ extension to the C bindings which delegate out to the C++ client code. So we might be way off 😆

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

調査の方向性

提供された Node.js スニペットを Pulsar 2.6.1 で再現し、続いて src/Consumer.cc における consumer の作成とシャットダウンを、特に参照されている 178 行目と 184 行目を中心に追跡します。この動作を ClientImpl による consumer と producer のクリーンアップと比較します。consumer と client を閉じた処理が AlreadyClosed エラーや未処理の promise rejection なしに完了すれば完了です。

索引モデルが issue の本文から書いたものです。

評価

技術スタック
cpp, javascript
領域
backend, distributed-systems
issue の種類
バグ
難易度
4/5
見積もり時間
3〜5日
活発さ
停滞
明瞭さ
おおむね明確
初心者へのやさしさ
35/100

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。