aws / aws/sagemaker-python-sdk

Make sourcedir.tar.gz and repacked model.tar.gz structure consistent

Đang mở
#3,491 0 bình luận 0 reaction 1 người được giao Được @nargokul nhận Xem trên GitHub
component: Inference APIs and Interfaces type: bug
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

When deploying a model together with customer code (described by one or more `Model.__init__` arguments among `entry_point`, `source_dir`, `dependencies`), the SDK (actually the relevant `Model.prepare_container_def` method) has 2 options for the customer code:
* **Either bundling all the code artifacts in a `sourcedir.tar.gz` file.** The file is then staged to S3 and later downloaded and extracted in the container at `/opt/ml/model/code`. If supplied, the `entry_point` file is copied at the root of the tar file. If supplied, the content of the `source_dir` directory is copied at the root of the tar file. If supplied, each dependency in `dependencies` is copied at the root of the tar file. This behavior is implemented by the `sagemaker.fw_utils.tar_and_upload_dir` function.
* **Or repacking the model and code artifacts together in a single `model.tar.gz` file.** The file is then staged to S3 and later downloaded by the container's host and made available in the container at `/opt/ml/model` where it is extracted. From the `model.tar.gz` file perspective, code artifacts (the `entry_point` file if supplied and the content of the `source_dir` directory if supplied) are placed in a `code` folder (location is relative to the root of the tar file). If supplied, each dependency in `dependencies` in placed in a `code/lib` folder. This behavior is implemented by the `sagemaker.utils._create_or_update_code_dir` function.

In both cases, code artifacts end up being available in the inference container at `/opt/ml/model/code`. However an inconsistency appears if we use `dependencies`. In that case, our `dependencies` end up being located:
* In `/opt/ml/model/code` if the code was bundled in a `source.dir.tar.gz` file.
* In `/opt/ml/model/code/lib` if the code was repacked with the model artifacts in a `model.tar.gz` file.

The SageMaker inference toolkits automatically add `/opt/ml/model` and `/opt/ml/model/code` to `sys.path`, unlike `/opt/ml/model/code/lib`. Therefore, `dependencies` located in the latter directory cannot be imported using the Python import system. The user/customer has to manually add this location to `sys.path` for its `dependencies` to be importable. This ultimately boils down to the inconsistency in the file structure which is annoying since the process of opting for a `sourcedir.tar.gz` or a repacked `model.tar.gz` is opaque to the user (and highly framework-dependent).

*Notice*: We do not consider the Multi-Model Enabled (MME) mode here.

IMHO, the solution with minimal impact would be not to create a `code/lib` directory in the case of the repacked `model.tar.gz`, `dependencies` would simply be copied to the `code` directory. Dependencies from a repacked `model.tar.gz` would then be directly available under `/opt/ml/code` which is already automatically added to `sys.path` by the inference toolkits. This solution would in fact simply align the structure of the repacked `model.tar.gz` file on the structure of the `sourcedir.tar.gz`. The latter being already in use, this fix should not raise backward-compatibility issues.

This topic directly relates to the following issues:
* [Issue 1065](https://github.com/aws/sagemaker-python-sdk/issues/1065) - Failed to import code copied into the /opt/ml/model/code/lib directory
* [Issue 1832](https://github.com/aws/sagemaker-python-sdk/issues/1832) - Extra lib directory when adding dependencies for PyTorchModel

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.