globus / globus/globus-compute
Multi-Queue Endpoints
- Dominant language
- Python
- Stars
- 162
- Forks
- 53
- Avg merge
- 15h 29m
- Merged PRs (30d)
- 26
Description
Currently each endpoint is started with a fixed Parsl configuration. This means that if the user wishes to use a new queue or change some aspect of the configuration on a live endpoint that isn't possible. Ideally, a single endpoint should be able to create
any number of "virtual endpoints" that represent pools of resources. Moving towards a model where the endpoint is started and
all endpoint management can be done over the funcX API would reduce friction for users.
Here's a sequence of capabilities we'll need to flesh out this feature:
- [ ] Allow endpoints to create, check_status and cancel "virtual endpoints"
- [ ] Add support for multiplexing virtual endpoints over a single channel to the forwarder
- [ ] Add message tags/headers to allow for routing tasks to the appropriate virtual endpoint
Contributor guide
Assessment
This issue has not been assessed yet.