apache / apache/iggy

Disk polling: measure task construction and completion payload costs

Open
#4,179 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
4.9k
Forks
432
Avg merge
2d 10h
Merged PRs (30d)
173

Description

Part of #4174, which records the shared profile revisions, workload and evaluation criteria.

[read_poll](https://github.com/apache/iggy/blob/8f1c0e260076f9fbe49539e904e43895bc24c3fe/core/shard/src/poll.rs#L241) transfers an owned `PollPlan` into a detached future, and [PollCompletionSender::complete](https://github.com/apache/iggy/blob/8f1c0e260076f9fbe49539e904e43895bc24c3fe/core/shard/src/poll/completion.rs#L161) constructs `Box`. Determine whether their current copying or allocation costs justify a representation change.

In the historical candidate's two captures without writes, `__memcpy_generic` samples with disk task creation as the first application caller accounted for **2.00 and 1.82 µs of CPU per completed poll**. Matching libc debug symbols resolved these samples. They identify copying during task creation, but neither the copied object nor the full construction cost or elapsed cost of boxing.

This is a lower priority lead: metric ownership and numeric history changes have since reduced carried state. Confirm that the old signal survives before selecting a change.

- Inspect optimized future and payload sizes, allocation counts and task creation caller stacks for an exact current release binary.
- Follow real ownership transfers through plan construction, execution, completion delivery and destruction. Account for copying, allocation placement and retained bytes throughout those lifetimes.
- Reuse the parent's diagnostics with and without writes. If a material cost remains, compare one representation change through completed polls using the shared timing and validation rules.

Preserve reservation release, cancellation behavior, late completion handling, history checks, owner acceptance and the disconnected requester guard. Removing boxing can enlarge futures or queue entries; pooling can increase retention. A standalone allocator benchmark omits these ownership and scheduling effects, and fewer allocations alone do not establish a useful improvement.

Contributor guide

Open the contributing guide

Research direction

Start at core/shard/src/poll.rs::read_poll and core/shard/src/poll/completion.rs::PollCompletionSender::complete. Inspect an optimized current release binary for future and payload sizes, allocation counts, and task-creation stacks, then reuse the parent diagnostics with and without writes. Done means any representation change is justified by material cost while preserving reservation release, cancellation, late completion, history checks, owner acceptance, and the disconnected requester guard.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, performance
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.