globus / globus/globus-compute

Multi-Queue Endpoints

Open
#186 0 comments 0 reactions 1 assignee Claimed by @yadudoc View on GitHub
enhancement Med:Pri
Dominant language
Python
Stars
162
Forks
53
Avg merge
15h 29m
Merged PRs (30d)
26

Description

Currently each endpoint is started with a fixed Parsl configuration. This means that if the user wishes to use a new queue or change some aspect of the configuration on a live endpoint that isn't possible. Ideally, a single endpoint should be able to create
any number of "virtual endpoints" that represent pools of resources. Moving towards a model where the endpoint is started and
all endpoint management can be done over the funcX API would reduce friction for users.

Here's a sequence of capabilities we'll need to flesh out this feature:

- [ ] Allow endpoints to create, check_status and cancel "virtual endpoints"
- [ ] Add support for multiplexing virtual endpoints over a single channel to the forwarder
- [ ] Add message tags/headers to allow for routing tasks to the appropriate virtual endpoint

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.