new feature: compio executor support
- Dominant language
- Rust
- Stars
- 5.4k
- Forks
- 825
- Avg merge
- 1d 14m
- Merged PRs (30d)
- 127
Description
### Feature Description
Add a new `Executor` under core that uses compio (compio-dispatcher, to be particular) to drive a future.
### Problem and Solution
There are 2 major obstacles:
- OpenDAL is dominently multithreaded, while compio is single threaded, meaning lots of the future returned by the runtime (include timeout) is `!Send + !Sync`, which is incompatible with lots of the traits OpenDAL have right now. This is in fact already solved in `service-compfs`, where `compio-dispatcher` is used to send an async task to be executed in a thread pool, so that caller will get a channel for returned value, hence `Send` and `Sync`, thread safe.
- `timeout` layer, with several other places where timeout is needed, currently assume tokio is used, and call `tokio::time::timeout/sleep` directly. This should be changed to a top-down invocation where all timeout/sleep call should be delegated all the way down to `Execute::timeout`. However the current `Executor` design makes this quite impossible. A change to `Executor`'s `timeout` function is needed.
### Additional Context
_No response_
### Are you willing to contribute to the development of this feature?
- [x] Yes, I am willing to contribute to the development of this feature.
Contributor guide
Research direction
Start by reading the existing Executor design under core and the compio-dispatcher usage in service-compfs. Trace the timeout layer and the Execute::timeout entry point to understand where Tokio is assumed. Done means a compio-backed Executor can drive futures safely and timeout/sleep calls are delegated through the executor design.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- infrastructure
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100