HDFGroup / HDFGroup/hermes

Improve performance for extremely small I/O workloads

Open
#680 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
34
Forks
19
PR merge metrics
No merged PRs in 30d

Description

Hermes performs well at 4KB, but not so well at 300 bytes -- which is the case for certain workload types we are encountering.

1. Make a lock-free hashmap supporting both inserts and deletes
2. Improve shared memory allocator performance by leveraging lock-free queues for a small cache
3. Allow clients to read data directly from the SHM hashmap

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by mapping Hermes’s shared-memory allocator, SHM hashmap, and I/O paths before evaluating the three proposed performance changes. Use the 300-byte workload as the baseline; the work is done when inserts and deletes are lock-free, small allocations use the proposed cache, and clients can read directly from the SHM hashmap with improved performance.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
backend, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.