fluent / fluent/fluent-logger-java

Best way to handle events that are too large for buffering

Ouverte
#56 1 commentaire 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Java
Étoiles
210
Forks
86
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

When looking through RawSocketSender to troubleshoot our other issue (see comment on #18), I noticed that RawSocketSender does not handle large events very well in send(). It assumes that flush() always makes enough capacity in the pendings buffer to save the new bytes. If the event was bigger than the buffer, you get a BufferOverflowException.

```
java.nio.BufferOverflowException
at java.nio.HeapByteBuffer.put(HeapByteBuffer.java:189)
at java.nio.ByteBuffer.put(ByteBuffer.java:859)
at org.fluentd.logger.sender.RawSocketSender.send(RawSocketSender.java:171)
```

Also, if every log message you send is larger than the buffer, then the RawSocketSender will attempt to flush to the socket on every message.

Is this the desired behavior? I could imagine adding a size check to the send() method that immediately returns false if the event is too large, to avoid the BufferOverflowException. Or perhaps the large event could skip the buffer and write directly out to the socket. Since we are already using a BufferedOutputStream, I also wonder why we need the internal pendings buffer at all?

What is the normal way of handling errors in RawSocketSender?

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Piste de recherche

Start in RawSocketSender.send(), especially the pendings buffer and the BufferOverflowException path shown in the stack trace; compare that behavior with the BufferedOutputStream usage. Determine and document the intended handling for events larger than the buffer and for sender errors, then verify that the chosen behavior is covered by tests.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
java
Domaine
backend
Type d'issue
Bug
Difficulté
5/5
Temps estimé
Plus d'une semaine
Activité
À l'abandon
Clarté
À clarifier
Accessibilité débutants
25/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.