apache / apache/pulsar-client-node
Interruptible Reader.readNext()?
- 主要言語
- C++
- スター
- 164
- フォーク
- 98
- PR マージ指標
- 30日以内にマージされた PR はありません
説明
Hi,
I have a use case where I expose a Pulsar topic over HTTP via Server-Sent Events. Basically, when a client connects over HTTP, I do this:
```js
const reader = await client.createReader({
topic: request.params.topic,
startMessageId: request.headers['last-event-id'] ?
Pulsar.MessageId.deserialize(Buffer.from(request.headers['last-event-id'], 'base64')) :
Pulsar.MessageId.earliest()
});
```
Then I use a loop that reads messages as they come and sends them to the client:
```js
while (!clientGoneAway) {
let message;
message = await reader.readNext();
reply.raw.write(formatServerSentEvent(
message.getMessageId().serialize().toString('base64'),
message.getData().toString('utf-8')
));
}
```
Additionally, the server detects the `close` event on the HTTP requests, closes the reader and prevents further iteration:
```js
request.raw.once('close', async function() {
clientGoneAway = true;
await reader.close();
reply.raw.end();
})
```
(`request.raw` and `reply.raw` are Node.js req and res objects, respectively - they're just wrapped like this in Fastify.js)
Now, my problem is that even if I call `reader.close()`, the `reader.readNext();` never resolves nor rejects. It's not just a Promise problem - it seems like it's keeping a thread busy, because then all other operations hang: things like `fs.createReadStream`, as well as creating new readers, hang forever until I completely restart the Node.js process.
I know I can use a timeout with `reader.readNext(timeoutMS)`, but this has 2 major disadvantages:
* It turns the reader into a kind of poller
* It still does not vacate the thread - so it's possible to trivially saturate the thread pool by creating more readers than the pool size within the timeout period (so for example 4 readers in 1 second, when using a timeout of 1000 ms)
Is there any way to have the reader immediately abort all reads when closed?
コントリビューションガイド
このリポジトリのコントリビューションガイドは索引されていません
調査の方向性
reader.readNext() と reader.close() が使用する Reader API のエントリポイントから始め、Node.js クライアントで読み取り中に close するケースを再現します。reader を閉じると保留中の読み取りが中断されるかを追跡し、関連する非同期動作を確認します。close 後にブロックされた読み取りが速やかに解決または拒否され、他の Node.js 操作を妨げなければ完了です。
索引モデルが issue の本文から書いたものです。
評価
- 技術スタック
- javascript, node.js
- 領域
- backend
- issue の種類
- バグ
- 難易度
- 4/5
- 見積もり時間
- 3〜5日
- 活発さ
- 停滞
- 明瞭さ
- おおむね明確
- 初心者へのやさしさ
- 38/100