opt: allgather extents on laminate
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 122
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
For apps that don't actually read anything until after a laminate, we can cache extents locally on each server until the laminate call. Then during the laminate, we could execute an allgather, for example, using a gather to root followed by a bcast. This could improve performance in gathering in two ways:
1) we can gather through a tree rather than using an N-to-1 pattern where each server sends directly to the root.
2) we could also merge extents as they travel up the tree rather than simply collecting extents to all be merged as the server.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the laminate call and extent-gathering behavior described in the issue. Trace where extents are collected and determine how local caching, a tree-based gather, and merging during the gather would fit. Done means the design is agreed, the implementation is tested, and the proposed gathering performance improvement is measured.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- distributed-systems, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100