apache / apache/pulsar-client-node
Interruptible Reader.readNext()?
- Lenguaje dominante
- C++
- Estrellas
- 164
- Forks
- 98
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
Hi,
I have a use case where I expose a Pulsar topic over HTTP via Server-Sent Events. Basically, when a client connects over HTTP, I do this:
```js
const reader = await client.createReader({
topic: request.params.topic,
startMessageId: request.headers['last-event-id'] ?
Pulsar.MessageId.deserialize(Buffer.from(request.headers['last-event-id'], 'base64')) :
Pulsar.MessageId.earliest()
});
```
Then I use a loop that reads messages as they come and sends them to the client:
```js
while (!clientGoneAway) {
let message;
message = await reader.readNext();
reply.raw.write(formatServerSentEvent(
message.getMessageId().serialize().toString('base64'),
message.getData().toString('utf-8')
));
}
```
Additionally, the server detects the `close` event on the HTTP requests, closes the reader and prevents further iteration:
```js
request.raw.once('close', async function() {
clientGoneAway = true;
await reader.close();
reply.raw.end();
})
```
(`request.raw` and `reply.raw` are Node.js req and res objects, respectively - they're just wrapped like this in Fastify.js)
Now, my problem is that even if I call `reader.close()`, the `reader.readNext();` never resolves nor rejects. It's not just a Promise problem - it seems like it's keeping a thread busy, because then all other operations hang: things like `fs.createReadStream`, as well as creating new readers, hang forever until I completely restart the Node.js process.
I know I can use a timeout with `reader.readNext(timeoutMS)`, but this has 2 major disadvantages:
* It turns the reader into a kind of poller
* It still does not vacate the thread - so it's possible to trivially saturate the thread pool by creating more readers than the pool size within the timeout period (so for example 4 readers in 1 second, when using a timeout of 1000 ms)
Is there any way to have the reader immediately abort all reads when closed?
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Línea de trabajo
Comienza en los puntos de entrada de Reader API utilizados por reader.readNext() y reader.close(), y reproduce el caso de cerrar mientras se está leyendo con el cliente de Node.js. Comprueba si cerrar el reader interrumpe una lectura pendiente y verifica el comportamiento asíncrono relacionado. Se considera completado cuando una lectura bloqueada se resuelve o se rechaza rápidamente después del cierre sin impedir otras operaciones de Node.js.
Escrito por el modelo de indexación a partir del texto del issue.
Evaluación
- Stack tecnológico
- javascript, node.js
- Área
- backend
- Tipo de issue
- Error
- Dificultad
- 4/5
- Tiempo estimado
- 3-5 días
- Estado de actividad
- Estancado
- Claridad
- Bastante claro
- Aptitud para principiantes
- 38/100