mars-project / mars-project/mars
MarsAPI should fetch data from worker directly and send data to client in a chunked manner
Open
@hekaisheng is already working on this.
Since Jul 6, 2019.
mod: web
type: enhancement
- Dominant language
- Python
- Stars
- 2.7k
- Forks
- 325
- PR merge metrics
- No merged PRs in 30d
Description
**Is your feature request related to a problem? Please describe.**
Currently, if fetching data on the client, the data will be fetched together into scheduler, and assembled there, then sent the total piece of data to client. It's extremely inefficient. MarsAPI should fetch data from worker directly one by one, the assemble job should be done one the client. For the web, the data should be sent in a chunked manner, instead of packing all data into the body.
**Additional context**
This is relative to #490 .
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.