googleapis / googleapis/python-aiplatform
Cannot Replace Model Deployed at PrivateEndpoint with a New Model Version
- Dominant language
- Python
- Stars
- 905
- Forks
- 465
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 44
Description
* Package version: 1.73.0
* Python 3.12.4
* MacOS 14.6.1; (also encountered the same problem on Vertex AI Pipeline run as a custom component)
I am deploying models to a PrivateEndpoint that attaches to a VPC network, using the following script.
```python
# already created PSA endpoint
endpoint = aiplatform.PrivateEndpoint(
endpoint_name=endpoint_resource_name
)
# already uploaded model
model = aiplatform.Model('projects/xxxxx/locations/us-central1/models/test_model_name@7')
# deploy new version to existing private endpoint
model.deploy(
endpoint=endpoint,
deployed_model_display_name=model_display_name,
machine_type=machine_type,
traffic_percentage=100,
min_replica_count=int(min_replica_count),
max_replica_count=int(max_replica_count),
network="projects/xxxxxx/global/networks/default",
)
```
or alternatively, I have also attempted to use `endpoint.deploy`
```python
endpoint.deploy(
model=model,
deployed_model_display_name=model_display_name,
machine_type=machine_type,
traffic_percentage=100,
min_replica_count=int(min_replica_count),
max_replica_count=int(max_replica_count),
)
```
When the endpoint was first created, this works fine. However, if there is a pre-existing model deployed, then deploying a new version of the model will throw the following error:
```
google.api_core.exceptions.FailedPrecondition: 400 There might be another deployed model or a failed deployed model that hasn't been cleaned up under the same private endpoint, please try again later or create another endpoint.
```
I also tried to use `gcloud ai` command line
```bash
gcloud ai endpoints deploy-model $ENDPOINT_NAME --project=$PROJECT_ID --region=us-central1 --model=$MODEL_NAME --display-name=$DISPLAY_NAME
```
The doc-string of the `PrivateEndpoint.deploy` method, the `traffic_percentage` states the following
```
traffic_percentage (int):
Optional. Desired traffic to newly deployed model.
Defaults to 0 if there are pre-existing deployed models.
Defaults to 100 if there are no pre-existing deployed models.
Defaults to 100 for PSA based private endpoint.
Negative values should not be provided. Traffic of previously
deployed models at the endpoint will be scaled down to
accommodate new deployed model's traffic.
Should not be provided if traffic_split is provided.
```
So it appears replacing a model with another model should have been supported. Setting `traffic_percentage` to 0 or commenting this argument out got me the same error as previously. I have also tried to set this `traffic_percentage` argument to `None`, and got
```
if traffic_percentage > 100:
^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```
Expected behavior: The PrivateEndpoint should be able to deploy new model version replacing the traffic of older model versions.
Other considerations (but maybe out of scope of the ticket): I need to access a MemoryStore Redis instance on GCP, and so far, only deploying the endpoint and Redis with PSA worked.
Contributor guide
Assessment
This issue has not been assessed yet.