nspcc-dev / nspcc-dev/neofs-node
Reuse object/data memory for replication workers
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 38
- Forks
- 51
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 38
Description
Is your feature request related to a problem? Please describe.
I'm always frustrated when I'm looking at the replication worker code. It takes and object from the storage, decompresses it, unmarshals it and then pushes the result to some other node. It allocates for the raw data, it allocates for the decompressed data, it allocates for the object and all of its fields. For big objects this means a heck of a lot of allocations.
Describe the solution you'd like
Have one per-replicator data buffer for raw data, one for decompressed data, one object. Reuse them. Likely this is not supported by our APIs at the moment, but this can probably be changed.
Additional context
#2300/#2178
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the replication worker code and the API paths used to retrieve, decompress, unmarshal, and forward objects. Review issues #2300 and #2178 for context; done means the worker reuses per-replicator raw-data, decompressed-data, and object storage without changing replication behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100