Limit number of partitions that data chunks of a "large" blob can reside on
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 1.8k
- Forks
- 298
- Avg merge
- 21h 43m
- Merged PRs (30d)
- 9
Description
Currently Ambry sprays the data chunks of a given "large" blob across all writable partitions (i.e. any partition is eligible to receive a data chunk). This can create some problems
1. If one chunk is lost, the whole blob is considered lost. Spraying it across multiple partitions spread over multiple nodes increases the area of failure
2. One disk being slow/unavailable can have a multiplying effect if it stores a single chunk of many large blobs
In order to restrict these effects, it may be useful to limit the number of partitions the data chunks of a particular "large" blob can reside on. At current time, GET operations request 4 chunks in parallel and maintain the number until the blob is currently served. It may be enough to have 4 * x partitions receive the data chunks (4,8,12 etc) to serve any parallelism requirements.
The partition picking can be random to start with and can be made more intelligent based on requirements (which need to be evaluated)
1. Pick from distinct disks and nodes
2. Pick from distinct disks but limit number of nodes across which these disks are spread (to lessen impact of node failures)
3. ...
Each of the options comes with its own upsides and downsides.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing how large-blob data chunks are assigned to writable partitions and how GET operations maintain four chunks in parallel. Evaluate the proposed partition limits and placement options against disk, node, and failure considerations. Done would require a decided design and an implemented, tested restriction on the partitions eligible for each large blob.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100