flashbots / flashbots/mosaik

Producer encoding errors: silent data loss and type mismatch

Open
#12 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
46
Forks
8
PR merge metrics
No merged PRs in 30d

Description

## Summary

Two related issues with how producer encoding errors are handled:

### 1. Silent data loss in fanout

In `src/streams/producer/worker.rs` (~line 224), when `Datum::encode()` fails during fanout, the error is logged and the datum is dropped. The caller of `producer.send()` has no way to know this happened.

Additionally, the dropped datum may be routed to the `undelivered` sink (~line 248), which is designed for "no consumers matched" — not "serialization failed." These two failure modes are conflated.

### 2. Error type hardcodes postcard

In `src/streams/producer/error.rs`, the `Encoding` variant uses the concrete `EncodeError` (postcard):

```rust
use crate::primitives::EncodeError;
// ...
Encoding(D, EncodeError),
```

But the `Datum` trait allows custom `EncodeError` types via associated types. The `Clone` impl reconstructs the error via `EncodeError::custom(e.to_string())`, which is lossy for non-postcard error types.

This breaks the abstraction that the customizable encoding feature was meant to provide.

## Suggestion

- Propagate encoding errors back to the `send()` caller, or provide an explicit error callback
- Keep the `undelivered` sink for "no match" only; handle encoding failures separately
- Make `Error` generic over `D::EncodeError` (or use a boxed error) instead of hardcoding postcard

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.