Typed exception for tuple deserialization failures
- Dominant language
- Java
- Stars
- 6.7k
- Forks
- 4k
- Avg merge
- 1d 7h
- Merged PRs (30d)
- 62
Description
The #9076 review raised a design question that deserves its own issue.
Today DeserializingConnectionCallback.recv() decides to drop a message by matching the decode failure against a list of exception types, walked along the cause chain. The review sketched a cleaner shape: KryoTupleDeserializer would validate the structural fields it already reads (source task id, stream id) and throw a dedicated exception for values the topology cannot resolve, instead of relying on a NullPointerException or IllegalArgumentException coming from deep in the lookup; the exception types that only a corrupt byte stream produces (KryoException, IOException, BufferUnderflowException, NegativeArraySizeException) would be wrapped into that same dedicated exception at the source; and recv() would catch exactly one intentional type, with everything else propagating.
The open problem is the wrap boundary. kryo.deserializeFrom runs the framework decode and the serializers registered through topology.kryo.register with nothing between them, so an IllegalArgumentException or ClassCastException thrown by a user serializer is indistinguishable from the same type thrown by the framework decode. Dropping both swallows real topology bugs. Unwrapping both is not possible either: with registration required, an unregistered class fails with a raw IllegalArgumentException from Kryo.getRegistration(Class), which is exactly the remote worker-kill vector #9074 fixed, so the generic types cannot simply be made fatal again.
Directions worth considering: structural validation typed at the source as above, plus a strict mode that turns drops back into fatal errors for operators who prefer loud failure; or attaching enough context (task, stream, input position) to the wrapped failure to separate framework decode errors from serializer errors in the logs.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with DeserializingConnectionCallback.recv() and KryoTupleDeserializer, then trace kryo.deserializeFrom and the topology.kryo.register serializers. Review #9074 and #9076 for the worker-kill and current exception-handling context. Done means an agreed wrap boundary and typed failure behavior that distinguishes intentional drops from serializer or framework errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100