googleapis / googleapis/python-aiplatform
`pipelinejob.get_associated_experiment()` sometimes incorrectly returns `None`
- Langage dominant
- Python
- Étoiles
- 905
- Forks
- 465
- Merge moyen
- 1 j 13 h
- PR mergées (30 j)
- 44
Description
#### Environment details
- OS type and version: MacOS and Linux
- Python version: 3.8.16
- pip version: 24.6
- `google-cloud-aiplatform` version: 1.36.0
#### Steps to reproduce
1. Create one or more Vertex Pipeline Job(s) with an experiment
2. Retrieve the job(s) using `aiplatform.PipelineJob.list()`
3. call `job.get_associated_experiment()` on any job
4. note that `None` is returned
5. do `print(job.task_details)` on a job
6. call `job.get_associated_experiment()` again on the same job
7. now the experiment is correctly returned
Step 5 can also be replaced with `job = aiplatform.PipelineJob.get(job.resource_name)` and the results will be the same. This is because `list()` does not populate the job's internal `_gca_resource`. However, `task_details` [calls](https://github.com/googleapis/python-aiplatform/blob/819e9ff568798232282e1f6dcf180cff3c788c63/google/cloud/aiplatform/pipeline_jobs.py#L748) `self._sync_gca_resource()` while `get()` [calls](https://github.com/googleapis/python-aiplatform/blob/819e9ff568798232282e1f6dcf180cff3c788c63/google/cloud/aiplatform/pipeline_jobs.py#L830) `self._gca_resource = self._get_gca_resource(resource_name=resource_name)`
#### Code example
```python
from google.cloud import aiplatform
def get_pipeline_jobs_with_experiment(
create_time_lower_bound,
project,
location,
experiment_name,
):
aiplatform.init(project=project, location=location)
filter = f'create_time > "{create_time_lower_bound}"'
print(f"Applying filter: {filter}")
# List pipeline jobs
pipeline_jobs = aiplatform.PipelineJob.list(
filter=filter,
order_by="create_time desc",
)
print(f"Found {len(pipeline_jobs)} pipeline jobs")
if experiment_name:
print(f"Filtering for experiment: {experiment_name}")
# Frustratingly get_associated_experiment() simply returns None (even when an experiment exists) when
# called on the output of PipelineJob.list() unless we first fetch each job via `get()`.
# It took a deep dive into the client library source code to figure this out.
# If this step is left out and the for loop is replaced with `for j in pipeline_jobs`
# we will get no associated experiments for all jobs
detailed_jobs = [
aiplatform.PipelineJob.get(job.resource_name) for job in pipeline_jobs
]
jobs_in_experiment = []
for j in detailed_jobs:
if not j.get_associated_experiment():
print(f"Job {j.name} has no associated experiment")
else:
print(f"Job {j.name} experiment: {j.get_associated_experiment().name}")
if j.get_associated_experiment().name == experiment_name:
jobs_in_experiment.append(j)
print(
f"Found {len(jobs_in_experiment)} pipeline jobs for experiment: {experiment_name}"
)
pipeline_jobs = jobs_in_experiment
return pipeline_jobs
```
### Suggested fix
If this is the intended behaviour it is not explained in either the `list()` or `get_associated_experiment()` method documentation. It is inconsistent: why does `task_details` automatically call `self._sync_gca_resource()` but `get_associated_experiment()` does not?
I suggest you add a `self._sync_gca_resource()` at the start of the `get_associated_experiment()` definition as well.
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.