globus / globus/globus-compute
Stream status information from endpoint to client
- Dominant language
- Python
- Stars
- 162
- Forks
- 53
- Avg merge
- 15h 29m
- Merged PRs (30d)
- 26
Description
**Is your feature request related to a problem? Please describe.**
Currently when a function is launched via the older API you get a function_id, and via the executor you get a future to track the progress of your function. However, in either case, there is no mechanism to get you the richer tasks status info that's available to the web-service as of #444. Users have expressed frustration about the lack of updates regarding the current state of the function within the function lifecycle.
**Describe the solution you'd like**
In the older system:
```
func_id = fxc.run(...)
try:
fxc.get_result(res)
except TaskPending as e:
print("Task is pending due to : ", e) # <- This will only say "waiting-for-ep"
```
In the newer executor:
```
fx_future = fx_executor.submit(...)
fx_future.done() -> # Returns a bool
# proposed:
fx_future.status() -> ['submitted', 'waiting-for-ep', 'dispatched-to-ep', 'waiting-for-resources', 'running', 'completed']
```
This task also requires mapping out the complete task lifecycle, as well as the states that we will be able to report.
To make this work, we'll need the following changes:
To support in the query API
1. Task status query to the web-service to return richer status info
2. client to report the status via the TaskPending exception? or via `fxc.get_status(func_id)`
To support via executor
1. Web-socket service to monitor task status changes via REDIS, we currently do not support streaming events via rabbit from web-service, although this could change.
2. Web-socket service to stream task status changes to client
3. Extend the executor to update status info via future.status field.
Contributor guide
Research direction
Start by mapping the complete task lifecycle and comparing the query API and executor paths, including Redis/RabbitMQ event handling and websocket status delivery. Define the reportable states and verify how the client should expose them through status querying, TaskPending, or future.status. Done means the chosen paths return the agreed lifecycle updates to clients.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rabbitmq, redis
- Domain
- api, backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100