aws / aws/sagemaker-python-sdk
ModelBuilder.deploy() should expose DataCacheConfig and other CreateInferenceComponent API parameters
- Langage dominant
- Python
- Étoiles
- 2.3k
- Forks
- 1.3k
- Merge moyen
- 1 j 22 h
- PR mergées (30 j)
- 35
Description
**Describe the feature you'd like**
Expose additional `CreateInferenceComponent` API parameters through `ModelBuilder.deploy()` when deploying Inference Components. Currently, `ModelBuilder._deploy_core_endpoint()` builds a minimal `InferenceComponentSpecification` and hardcodes several values. The following API-supported configuration is not surfaced:
- **`Specification.DataCacheConfig.EnableCaching`** — Cache model artifacts and container images on instances for faster auto-scaling cold starts
- **`Specification.BaseInferenceComponentName`** — Adapter component deployment (e.g. LoRA adapters attached to a base model)
- **`Specification.Container`** (`Image`, `ArtifactUrl`, `Environment`) — Custom container images, artifact URLs, and environment variables at the IC level
- **`VariantName`** (top-level) — Currently hardcoded to `"AllTraffic"`, not configurable for multi-variant endpoints
**How would this feature be used? Please describe.**
Our immediate need is `DataCacheConfig.EnableCaching`. When deploying large model artifacts (700MB+ Triton ensembles) on auto-scaling endpoints, new instances must re-download all artifacts during scale-out. Enabling caching eliminates this overhead.
Ideally these would be optional parameters on `deploy()` or on `ResourceRequirements`:
```python
builder.deploy(
endpoint_name="my-endpoint",
inference_component_name="my-ic",
instance_type="ml.g5.2xlarge",
initial_instance_count=1,
inference_config=ResourceRequirements(
requests={"memory": 8192, "num_accelerators": 1, "num_cpus": 2, "copies": 1}
),
data_cache_config={"enable_caching": True}, # new
)
```
**Describe alternatives you've considered**
The current workaround is to use `ModelBuilder.build()` for model creation, then call `EndpointConfig.create()`, `Endpoint.create()`, and `InferenceComponent.create()` directly via sagemaker-core, which supports the full `InferenceComponentSpecification`:
```python
from sagemaker.core.resources import EndpointConfig, Endpoint, InferenceComponent
from sagemaker.core.shapes import (
ProductionVariant,
InferenceComponentSpecification,
InferenceComponentComputeResourceRequirements,
InferenceComponentRuntimeConfig,
InferenceComponentDataCacheConfig,
)
builder.build(model_name=sm_model_name)
EndpointConfig.create(
endpoint_config_name=epc_name,
production_variants=[
ProductionVariant(
variant_name="AllTraffic",
instance_type="ml.g5.2xlarge",
initial_instance_count=1,
)
],
execution_role_arn=role,
)
Endpoint.create(
endpoint_name=endpoint_name,
endpoint_config_name=epc_name,
)
InferenceComponent.create(
inference_component_name=ic_name,
endpoint_name=endpoint_name,
variant_name="AllTraffic",
specification=InferenceComponentSpecification(
model_name=sm_model_name,
compute_resource_requirements=InferenceComponentComputeResourceRequirements(
min_memory_required_in_mb=8192,
number_of_accelerator_devices_required=1,
number_of_cpu_cores_required=2,
),
data_cache_config=InferenceComponentDataCacheConfig(enable_caching=True),
),
runtime_config=InferenceComponentRuntimeConfig(copy_count=1),
)
```
This works but defeats the purpose of `ModelBuilder.deploy()` as a high-level abstraction. Customers shouldn't need to drop down to sagemaker-core or boto3 for commonly used API parameters.
**Additional context**
- API reference: [CreateInferenceComponent](https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateInferenceComponent.html)
- The shape classes (`InferenceComponentDataCacheConfig`, `InferenceComponentContainerSpecification`, etc.) already exist in `sagemaker.core.shapes` — they just need to be wired into `ModelBuilder._deploy_core_endpoint()`.
- SageMaker SDK version: `3.x`
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez dans ModelBuilder.deploy() et ModelBuilder._deploy_core_endpoint(), puis examinez les classes InferenceComponentSpecification existantes et les classes shape associées dans sagemaker.core.shapes. Suivez la manière dont les paramètres de CreateInferenceComponent sont assemblés et identifiez les tests de deployment existants du projet. Le travail est terminé lorsque les paramètres optionnels de data cache, base component, container et variant peuvent être transmis au lieu d’être omis ou codés en dur.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- aws, python
- Domaine
- api, cloud, machine-learning
- Type d'issue
- Fonctionnalité
- Difficulté
- 4/5
- Temps estimé
- 3-5 jours
- Activité
- Calme
- Clarté
- Plutôt claire
- Accessibilité débutants
- 68/100