new feature: remove the remaining AsyncWrite adapter copy from Python AsyncFile.write
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Pass immutable Python bytes to the core async Writer without first copying them through the FuturesAsyncWriter staging buffer.
### Problem and Solution
`AsyncFile.write` already retains immutable Python input with `PyBackedBytes`, but then calls `FuturesAsyncWriter.write_all(&bs)`. That adapter copies borrowed input into 256 KiB FlexBuf blocks. The core writer subsequently owns storage chunking.
The binding could retain the core `Writer` and reuse `py_bytes_like_into_buffer`, already used for owner-preserving whole-object writes, to pass an owned Buffer directly. Preserve small-write coalescing rather than mapping every small Python write to a storage request.
Acceptance criteria:
- Preserve write-all byte counts, accepted immutable input types, context-manager completion, cancellation, and close errors.
- Keep Python owners alive through buffered/in-flight/retried operations.
- Verify tiny, exact-part, non-multiple, and mixed-sized writes with full readback.
- Measure removed copy/allocation work and retained owners, then validate real file uploads without weakening SDK integrity or changing the workload.
### Additional Context
Follow-up to #8159 and merged #8171. That PR removed the initial Python-bytes-to-Vec copy and explicitly retained the 256 KiB writer adapter. This request targets the remaining adapter copy, not the already-fixed copy.
[AsyncFile state and write](https://github.com/apache/opendal/blob/b6cf44f7b8a1523409e0e998e478c996ac970f03/bindings/python/src/file.rs#L430); [Existing bytes owner conversion](https://github.com/apache/opendal/blob/b6cf44f7b8a1523409e0e998e478c996ac970f03/bindings/python/src/utils.rs#L29); [FuturesAsyncWriter buffering](https://github.com/apache/opendal/blob/b6cf44f7b8a1523409e0e998e478c996ac970f03/core/core/src/types/write/futures_async_writer.rs#L35)
The synchronous File flush contract should be evaluated separately; this request is scoped to AsyncFile.
Source references are pinned to `b6cf44f7b8a1523409e0e998e478c996ac970f03`. This request describes an optimization opportunity; it does not claim a measured end-to-end speedup.
Contributor guide
Research direction
Start with AsyncFile.write in bindings/python/src/file.rs and compare its use of py_bytes_like_into_buffer in bindings/python/src/utils.rs with FuturesAsyncWriter in core/core/src/types/write/futures_async_writer.rs. Trace ownership, buffering, cancellation, close errors, and retry behavior before evaluating the scoped AsyncFile change. Done means the listed write patterns, readback, owner retention, real uploads, and copy/allocation measurements pass without weakening SDK integrity.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100