googleapis / googleapis/python-aiplatform

`pipelinejob.get_associated_experiment()` sometimes incorrectly returns `None`

Aperta
#4,341 0 commenti 0 reazioni 1 assegnatario Rivendicata da @lakeyk Vedi su GitHub
api: vertex-ai
Lingua principale
Python
Stelle
905
Fork
465
Merge medio
1g 13h
PR unite (30g)
44

Descrizione

#### Environment details

- OS type and version: MacOS and Linux
- Python version: 3.8.16
- pip version: 24.6
- `google-cloud-aiplatform` version: 1.36.0

#### Steps to reproduce

1. Create one or more Vertex Pipeline Job(s) with an experiment
2. Retrieve the job(s) using `aiplatform.PipelineJob.list()`
3. call `job.get_associated_experiment()` on any job
4. note that `None` is returned
5. do `print(job.task_details)` on a job
6. call `job.get_associated_experiment()` again on the same job
7. now the experiment is correctly returned

Step 5 can also be replaced with `job = aiplatform.PipelineJob.get(job.resource_name)` and the results will be the same. This is because `list()` does not populate the job's internal `_gca_resource`. However, `task_details` [calls](https://github.com/googleapis/python-aiplatform/blob/819e9ff568798232282e1f6dcf180cff3c788c63/google/cloud/aiplatform/pipeline_jobs.py#L748) `self._sync_gca_resource()` while `get()` [calls](https://github.com/googleapis/python-aiplatform/blob/819e9ff568798232282e1f6dcf180cff3c788c63/google/cloud/aiplatform/pipeline_jobs.py#L830) `self._gca_resource = self._get_gca_resource(resource_name=resource_name)`

#### Code example

```python

from google.cloud import aiplatform

def get_pipeline_jobs_with_experiment(
create_time_lower_bound,
project,
location,
experiment_name,
):
aiplatform.init(project=project, location=location)

filter = f'create_time > "{create_time_lower_bound}"'

print(f"Applying filter: {filter}")

# List pipeline jobs
pipeline_jobs = aiplatform.PipelineJob.list(
filter=filter,
order_by="create_time desc",
)

print(f"Found {len(pipeline_jobs)} pipeline jobs")

if experiment_name:
print(f"Filtering for experiment: {experiment_name}")

# Frustratingly get_associated_experiment() simply returns None (even when an experiment exists) when
# called on the output of PipelineJob.list() unless we first fetch each job via `get()`.
# It took a deep dive into the client library source code to figure this out.
# If this step is left out and the for loop is replaced with `for j in pipeline_jobs`
# we will get no associated experiments for all jobs
detailed_jobs = [
aiplatform.PipelineJob.get(job.resource_name) for job in pipeline_jobs
]
jobs_in_experiment = []
for j in detailed_jobs:
if not j.get_associated_experiment():
print(f"Job {j.name} has no associated experiment")
else:
print(f"Job {j.name} experiment: {j.get_associated_experiment().name}")
if j.get_associated_experiment().name == experiment_name:
jobs_in_experiment.append(j)

print(
f"Found {len(jobs_in_experiment)} pipeline jobs for experiment: {experiment_name}"
)

pipeline_jobs = jobs_in_experiment

return pipeline_jobs
```

### Suggested fix

If this is the intended behaviour it is not explained in either the `list()` or `get_associated_experiment()` method documentation. It is inconsistent: why does `task_details` automatically call `self._sync_gca_resource()` but `get_associated_experiment()` does not?

I suggest you add a `self._sync_gca_resource()` at the start of the `get_associated_experiment()` definition as well.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.