nspcc-dev / nspcc-dev/neofs-node
Write replicated data directly to disk
Open
Nobody has claimed this yet.
enhancement
I3
neofs-storage
performance
S3
U2
- Dominant language
- Go
- Stars
- 38
- Forks
- 51
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 38
Description
Is your feature request related to a problem? Please describe.
I'm always frustrated when we're buffering whole object payload before writing it to disk. While current replication API doesn't give any other option (single message needs to be received completely), https://github.com/nspcc-dev/neofs-api/issues/416 changes it.
Describe the solution you'd like
- Open a temp file in target shard.
- Verify header.
- Write to this file directly, calculate checksum on the go.
- If it's all good, finalize the write operation, update metabase, etc.
- If it's not --- drop temporary file.
Describe alternatives you've considered
Not a lot of them.
Additional context
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the current replication API and the target-shard write path, then review the linked neofs-api issue and neofs-node issue #3918. Done means replicated payloads are written to a temporary shard file while the header is verified and a checksum is calculated, then finalized with metabase updates or discarded on failure.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100