alibaba / alibaba/feathub

Add test case to verify SparkJob completed exceptionally

Open
#233 0 comments 0 reactions 0 assignees View on GitHub
type:improvement
Dominant language
Python
Stars
350
Forks
60
PR merge metrics
No merged PRs in 30d

Description

```python
class SparkJob(ProcessorJob):
"""Represent a Spark job."""

def __init__(
self,
job_future: Future,
) -> None:
super().__init__()
self._job_future = job_future

# TODO: Add test case to verify this method's behavior when job future
# is completed exceptionally.
def cancel(self) -> Future:
cancel_future: Future = Future()
job_future_callback = self._get_job_future_callback(cancel_future)
self._job_future.add_done_callback(job_future_callback)
return cancel_future
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at SparkJob.cancel and inspect the existing SparkJob tests and _get_job_future_callback behavior. Add a test covering an exceptionally completed job future, then run the relevant Python test suite and confirm the returned cancel future reflects the expected exceptional result.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, spark
Domain
data-engineering
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.