HDFGroup / HDFGroup/hermes

Support mmap/sbrk interception for avoiding data copies between runtime and client

Open
#581 0 comments 0 reactions 0 assignees View on GitHub
priority-medium
Dominant language
C++
Stars
34
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Currently, we copy data between clients and runtime since the runtime is in a different address space. However, if we intercept mmap/sbrk and put all data in SHM, this will allow applications to avoid the data copy, particularly for GET.

Each process will have to register its private allocator with the runtime. The runtime will have to store a mapping between process ID and allocator.

A well-formed version will have to have a wreaper thread to unregister dead processes automatically.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by tracing the runtime and client allocation paths involved in copying data, then examine how mmap/sbrk interception and shared memory could fit those paths. Identify where processes would register private allocators and where the runtime would maintain the process-to-allocator mapping. Done means data can avoid copies, registrations are tracked, and a wreaper thread unregisters dead processes.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
operating-systems, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.