Workarounds and visibility on IOException: Message too long
- Dominant language
- Java
- Stars
- 100
- Forks
- 100
- PR merge metrics
- No merged PRs in 30d
Description
We are running into the well known (#4) problem that sometimes a segment is too big to fit in a UDP message (65,507 bytes). For example because it contains a big sql statement or an exception.
The problem is however that if these issues happen sporadically they are very difficult to debug as there is almost no visibility.
We see a few possible improvements which can be made easily:
* _Enhance visibility_: For example by logging the segment when it is too big.
It could be as easy as doing something like this:
```diff
}
- private boolean sendData(byte[] data) {
+ private boolean sendData(byte[] data, Entity entity) {
+ if (data.length > 65_527) {
+ logger.warn("(sub)segment is too big to send: " + entity.prettySerialize());
+ return false;
+ }
DatagramPacket packet = new DatagramPacket(sendBuffer, DAEMON_BUF_RECEIVE_SIZE, config.address);
```
(Btw: it also puzzles us why `DAEMON_BUF_RECEIVE_SIZE` is a lot bigger then 65_527)
* _Add retry behavior_ which tries to send the completed subsegments (so if one is too big, it does not break the others). We for example currently quickly implemented a `RetryingEmitter` which works like this:
```java
public class RetryingEmitter extends UDPEmitter {
private static final Log logger = LogFactory.getLog(MethodHandles.lookup().lookupClass());
private final StreamingStrategy streamingStrategy;
public RetryingEmitter(StreamingStrategy streamingStrategy) throws SocketException {
super();
this.streamingStrategy = streamingStrategy;
}
public boolean sendSegment(Segment segment) {return trySplitOnFailure(segment, super::sendSegment);}
public boolean sendSubsegment(Subsegment subsegment) {return trySplitOnFailure(subsegment, super::sendSubsegment);}
boolean trySplitOnFailure(T entity, Predicate send) {
boolean ok = send.test(entity);
if (!ok) {
logger.info("send of entity " + entity.getId() + " failed, retrying.");
streamingStrategy.streamSome(entity, this);
ok = send.test(entity);
if (!ok) {
logger.warn("Unable to send entity " + entity.getId() + " after split. size: " +
entity.serialize().length() + ", content: " + entity.prettySerialize());
}
}
return ok;
}
}
```
This is of course rather naive, as a lot of errors are still send. But it helps us to debug these kinds of issues.
What do you think about these proposals?
Thanks,
Steven
Contributor guide
Assessment
This issue has not been assessed yet.