AgnostiqHQ / AgnostiqHQ/covalent-awslambda-plugin
Support task cancellation
- Dominant language
- Python
- Stars
- 8
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
## Description
Covalent now supports dispatch cancellation, as a result all executor plugins can opt-in to this functionality by exposing a `cancel` method, periodically checking if cancellation has been requested and registering a `job handle` (job ID) associated with a task (in this case the function name or function ARN).
Essentially only the following methods are defined in the parent classes of the executors need to be consumed within the executor class:
- `await self.get_cancel_requested()` in order to determine if task cancellation has been requested, at which point it is appropriate to raise a `TaskCancelledError` exception
- `await self.set_job_handle(handle=job_handle)` should be set once the `job handle` is known
- expose a `async def cancel(self, task_metadata: Dict, job_handle: str)` method in the executor class
The below code can be used as a reference:
```python
from covalent._shared_files.exceptions import TaskCancelledError
...
async def proceed_if_task_not_cancelled(self):
if await self.get_cancel_requested():
self._debug_log(f"Task Cancelled")
raise TaskCancelledError(f"Batch job {batch_job_name} requested to be cancelled")
async def run(self, function: Callable, args: List, kwargs: Dict, task_metadata: Dict) -> Any:
...
await self.proceed_if_task_not_cancelled()
# pickle task
...
await self.proceed_if_task_not_cancelled()
# upload pickled assets
...
await self.proceed_if_task_not_cancelled()
# invoke job/task
await self.set_job_handle(handle=job_handle)
async def cancel(self, task_metadata: Dict, job_handle: str) -> None:
"""
Cancel the batch job
Arg(s)
task_metadata: Dictionary with the task's dispatch_id and node id
job_handle: Unique job handle assigned to the task by Batch
Return(s)
None
"""
# boto client invocations to cancel the task
```
## Acceptance Criteria
- [ ] Update executor to listen for cancellation requested periodically
- [ ] Set `job handle` once job / task id is determined
- [ ] Ensure that the a workflow is tested with cancellation to ensure `cancel` functionality correctly integrated
- [ ] Expose `cancel` method (may not be possible with lambda so the above tasks suffice)
Contributor guide
Research direction
Locate the executor class and its parent cancellation methods, then trace the run path to identify where cancellation checks and the job handle belong. Verify the workflow with cancellation, confirm the handle is registered once the task ID is known, and expose cancel where AWS Lambda permits it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100