googleapis / googleapis/python-aiplatform

CustomContainerTrainingJob.run drops max_wait_duration=0 instead of requesting indefinite DWS wait

Đang mở Phù hợp với người mới
#7,067 1 bình luận 0 reaction 0 người được giao Xem trên GitHub
api: vertex-ai
Ngôn ngữ chính
Python
Star
905
Fork
465
Merge trung bình
1 ngày 13 giờ
Pull request đã merge (30 ngày)
44

Mô tả

Thanks for maintaining this library. `CustomContainerTrainingJob.run(max_wait_duration=0)` drops the value because the high-level training-job implementation uses a truthiness check. This prevents callers from requesting the documented indefinite wait for DWS Flex Start.

#### Environment details

- OS type and version: macOS
- Python version: 3.12
- `google-cloud-aiplatform` versions: reproduced in 1.148.1; source inspection confirms the same behavior in 1.150.0, 1.163.0, and current `main`

#### Steps to reproduce

1. Create a `CustomContainerTrainingJob` using any valid project, staging bucket, and container image.
2. Run it with Flex Start and `max_wait_duration=0`.
3. Inspect the scheduling inputs produced by `_prepare_training_task_inputs_and_output_dir` or the resulting `TrainingPipeline` request.

#### Code example

```python
job.run(
scheduling_strategy=custom_job.Scheduling.Strategy.FLEX_START,
max_wait_duration=0,
)
```

Observed scheduling value:

```text
max_wait_duration: None
```

Expected scheduling value:

```text
max_wait_duration: 0s
```

For comparison, a positive input such as `14400` is preserved as `14400s`.

#### Cause

The high-level training-job path converts the duration using a truthiness check, so zero follows the same branch as `None`:

https://github.com/googleapis/python-aiplatform/blob/v1.163.0/google/cloud/aiplatform/training_jobs.py#L1663-L1667

The generated API contract says that explicit zero means indefinite waiting, while omission defaults to 24 hours:

https://github.com/googleapis/python-aiplatform/blob/v1.163.0/google/cloud/aiplatform_v1/types/custom_job.py#L577-L582

`max_wait_duration` is a presence-aware `google.protobuf.Duration`, so explicit zero and an omitted field are distinct requests.

A previous fix correctly changed the lower-level `CustomJob` and hyperparameter-tuning paths to use `is not None`, explicitly noting that `0` is valid:

https://github.com/googleapis/python-aiplatform/commit/d9675fdf051233539f478187143f2833fd6e6af0

That commit did not update the `CustomContainerTrainingJob` path in `training_jobs.py`.

#### Suggested fix

Use an explicit `is not None` check in `_prepare_training_task_inputs_and_output_dir`, consistent with the lower-level implementation, and add tests covering omitted, zero, and positive durations.

#### Stack trace

No exception is raised. The problem is a silent request-semantics change: `0` is serialized as omission.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Hướng nghiên cứu

Start in google/cloud/aiplatform/training_jobs.py at _prepare_training_task_inputs_and_output_dir, then compare the lower-level CustomJob implementation referenced in the issue. Verify the scheduling inputs for omitted, zero, and positive max_wait_duration values; done means zero is serialized as 0s while omission remains distinct and positive durations are preserved.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
python
Lĩnh vực
machine-learning
Loại issue
Lỗi
Độ khó
2/5
Thời gian dự kiến
1-3 giờ
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Đặc tả rõ ràng
Mức phù hợp với người mới
78/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.