bug: etcd reads larger than 4 MiB fail at the client decoding limit
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
## OpenDAL version or commit
PR #8119 head `b838b5fadb8eb5e53b77ae75bf574eef2914488e`
(OpenDAL 0.58.2 in the reproduced CI job).
## Affected service or component
`services/etcd` against etcd v3.6.4.
## Describe the bug
The etcd service can write an 8 MiB value when the server allows it, but reading
the value fails because etcd-client 0.19.0 applies a default 4 MiB gRPC decoding
limit to KV responses.
This is a client response-size limit, not a write-size limit, so
`Capability::write_total_max_size` cannot describe it correctly.
## Steps to Reproduce
Start the repository's etcd fixture:
```bash
docker compose -f fixtures/etcd/docker-compose-standalone.yml up -d --wait
export OPENDAL_ETCD_ENDPOINTS=http://127.0.0.1:2379
export OPENDAL_ETCD_ROOT=/tmp/opendal
export OPENDAL_TEST=etcd
cd bindings/dotnet
cargo build
dotnet test -f net10.0 \
--filter 'FullyQualifiedName~ReadBehavior_LargePayload_RoundTripsExactly'
```
The fixture configures etcd with `--max-request-bytes=10485760`.
The failure is also reproduced in:
https://github.com/apache/opendal/actions/runs/32733492592/job/97459019848
## Actual Behavior
Writing the 8 MiB value succeeds. Reading it fails with:
```text
decoded message length too large: found 8388766 bytes, the limit is: 4194304 bytes
```
## Expected Behavior
The 8 MiB round trip succeeds when the etcd server is configured to accept the
payload and the client decoding limit is configured accordingly.
## Additional Context
etcd-client exposes `KvClient::max_decoding_message_size`, but OpenDAL's etcd
configuration currently provides no way to set it.
A fix should:
- Add an explicit etcd client decoding-size configuration without defaulting to
an unbounded response size.
- Apply the configured limit to KV reads.
- Configure the repository's etcd fixture with a limit large enough for the
existing 8 MiB behavior test.
- Remove the temporary etcd-specific skip after the behavior test passes.
Contributor guide
Research direction
Start in services/etcd and inspect the etcd client setup, then run the repository fixture from fixtures/etcd/docker-compose-standalone.yml and the ReadBehavior_LargePayload_RoundTripsExactly test. Add a bounded decoding-size configuration, apply it to KV reads, configure the fixture for the 8 MiB payload, and remove the temporary etcd-specific skip when the round trip passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, rust
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 70/100