AgnostiqHQ / AgnostiqHQ/covalent-awslambda-plugin
Update Lambda executor to return stdout, stderr, and runtime exceptions
- Dominant language
- Python
- Stars
- 8
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
The purpose of this issue is to ensure that the Lambda executor
- returns any stdout and stderr printed by tasks using the mechanism defined in [#1380](https://github.com/AgnostiqHQ/covalent/issues/1380). To do this, the `run()` implementation needs to
- retrieves the `stdout` and `stderr` from the executor backend -- in the case of AWS Executors, by parsing Cloudwatch logs (see how [Braket does this](https://github.com/AgnostiqHQ/covalent-braket-plugin/blob/874bc6e09dc8ab5504edfb024f8df5eefa08978a/covalent_braket_plugin/braket.py#L330))
- printing those strings to `self.task_stdout` and `self.task_stderr`.
- distinguish task runtime exceptions -- those raised in the executor backend -- from other exceptions originating from the interaction of Covalent and the executor backend. This is explained in more detail in [#1390](https://github.com/AgnostiqHQ/covalent/issues/1390). When an runtime exception occurs, the `run()` implementation should:
1. Retrieve whatever `stdout` and `stderr` messages have already been printed by the task
2. Ensure that the exception traceback is appended to the task's `stderr`.
3. Print `stdout` and `stderr` to `self.task_stdout` and `self.task_stderr`, respectively.
4. Raise a `TaskRuntimeException`.
For examples, see how the [dask executor now deals with task runtime exceptions](https://github.com/AgnostiqHQ/covalent/blob/75b97f8d4630a324848ab446f48c1c9911a248df/covalent/executor/executor_plugins/dask.py#L109).
**Note**: These changes should be implemented in a backward-compatible manner -- so that the new AWSExecutors should work with Covalent 0.202.0post1 (the current stable AWSExecutors work with the latest develop).
### Acceptance criteria:
- [ ] Any stdout and stderr printed by a task before raising an unhandled exception is retrieved and printed to `self.task_stdout` and `self.task_stderr` respectively, where `self` is the executor plugin instance.
- [ ] If a task raises an exception:
- [ ] The traceback is included in the task’s stderr.
- [ ] The `run()` method raises a `TaskRuntimeError`.
- [ ] The executor plugin remains compatible with Covalent Core `0.202.0post1.
Contributor guide
Research direction
Start with the Lambda executor's run() implementation and compare the backend behavior with covalent/executor/executor_plugins/dask.py and the referenced covalent_braket_plugin/braket.py example. Verify how stdout, stderr, and runtime tracebacks are retrieved and exposed, then check that the resulting behavior remains compatible with Covalent Core 0.202.0post1 and meets the listed acceptance criteria.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, python
- Domain
- backend, cloud
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100