googleapis / googleapis/python-aiplatform

Add date/time filtering capability to Vertex AI Experiment.get_data_frame API to prevent 503 errors with large runs

未关闭
#5,298 0 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看
api: vertex-ai
主要语言
Python
星标
905
派生
465
平均合并
1 天 13 小时
30 天内合并 PR
44

描述

# Problem
When attempting to use Experiment.get_data_frame() with projects containing numerous runs, the API consistently returns 503 "Service Unavailable" errors. This appears to happen because the API attempts to fetch all experiment runs at once, which can overload the service when dealing with large datasets.

# Current behavior
``` python
experiment = aiplatform.Experiment(experiment_name="specific-experiment-name")
df_experiment = experiment.get_data_frame(include_time_series=False)
```
This call fails with 503 errors when the experiment contains too many runs.

# Proposed solution
Add date/time filtering parameters to the get_data_frame() method to limit the number of runs being processed:
``` python
df_run = run.get_data_frame(
include_time_series=False,
start_time="2023-01-01T00:00:00Z",
end_time="2023-01-31T23:59:59Z"
)
```
This would allow users to retrieve manageable chunks of experiment data by time period, preventing service overload and improving reliability when working with large experiments.

# Additional benefits
- Improved performance for large experiments
- Better user experience by avoiding timeout errors
- More control over data retrieval for analysis purposes
- Reduced load on backend services

贡献指南

打开贡献指南

调研方向

Start at the Vertex AI Experiment.get_data_frame() entry point and trace how experiment runs are retrieved. Compare the proposed start_time and end_time arguments with the current example, which also refers to run.get_data_frame(). Done means callers can limit retrieval to a date/time range and large experiments avoid the reported 503 failure.

由索引模型根据 Issue 内容生成。

评估

技术栈
google-cloud, python
领域
api, cloud, machine-learning
Issue 类型
功能
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
45/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。