apache / apache/brpc

Does IOPortal consider to support to create memory aligned block for direct IO

Open
#993 1 comment 0 reactions 0 assignees View on GitHub
feature
Dominant language
C++
Stars
17.6k
Forks
4.1k
Avg merge
2d 12h
Merged PRs (30d)
69

Description

1. IOPortal calls `pappend_from_file_descriptor` to read data from a opened `fd`, where in my condition `fd` is opened with `O_DIRECT` flag, but the function failed with `invalid argument`. I am sure that the offset and size are chunk-aligned, so I guess this is because the `memory buffer` is not chunk-aligned.
2. I read `iobuf.cpp`, it call `blockmem_allocate` (default is `::malloc`) to alloc memory for `create_block`, which is not memory-aligned for `direct io` operations. This may be the reason for invalid argument.
3. Do we have plan for memory-aligned block allocation or in deed is there a way to achieve this goal. I want to minimize the data copy as fewer as possible.

Contributor guide

Open the contributing guide

Research direction

Read iobuf.cpp and trace pappend_from_file_descriptor through create_block and blockmem_allocate. Check how the memory buffer is allocated for O_DIRECT reads and whether the existing path exposes alignment requirements. Done means documenting or implementing a supported aligned-allocation path that avoids unnecessary copying, with its behavior verified for direct I/O.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.