aws / aws/sagemaker-python-sdk
`FrameworkProcessor._package_code` fails on Windows with `PermissionError` (WinError 32) when deleting temp tar.gz
- Dominant language
- Python
- Stars
- 2.3k
- Forks
- 1.3k
- Avg merge
- 1d 22h
- Merged PRs (30d)
- 35
Description
**PySDK Version**
- [ ] PySDK V2 (2.x)
- [x] PySDK V3 (3.x)
**Describe the bug**
On Windows, calling `FrameworkProcessor.run()` with a local `source_dir` fails during code packaging with:
```
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\...\\AppData\\Local\\Temp\\tmpXXXX.tar.gz'
```
The failure occurs in `FrameworkProcessor._package_code()` at `os.unlink(tmp.name)` ([`sagemaker-core/src/sagemaker/core/processing.py`](https://github.com/aws/sagemaker-python-sdk/blob/master/sagemaker-core/src/sagemaker/core/processing.py) ~L1159–L1182).
The method creates a temp file with `tempfile.NamedTemporaryFile(..., delete=False)` and calls `os.unlink(tmp.name)` **inside** the `with` block, while the `NamedTemporaryFile` handle is still open. On Windows, a file cannot be deleted while any handle remains open; on Linux/macOS this pattern often succeeds.
**To reproduce**
1. Use Windows (tested on Windows 11).
2. Install PySDK V3, e.g. `pip install sagemaker-core` (tested with `sagemaker-core==2.11.0`).
3. Save and run the script below (no AWS credentials required; S3 upload is mocked):
```python
"""Minimal repro for FrameworkProcessor._package_code WinError 32 on Windows."""
import os
import sys
from unittest.mock import patch
if sys.platform != "win32":
print("Skip: this bug only reproduces on Windows.")
sys.exit(0)
from sagemaker.core.helper.session_helper import Session
from sagemaker.core.processing import FrameworkProcessor
SOURCE_DIR = "minimal_src"
os.makedirs(SOURCE_DIR, exist_ok=True)
with open(os.path.join(SOURCE_DIR, "script.py"), "w", encoding="utf-8") as f:
f.write("print('hello')\n")
sess = Session()
processor = FrameworkProcessor(
image_uri="123456789012.dkr.ecr.us-east-1.amazonaws.com/pytorch-training:1.0.0-cpu-py3",
role="arn:aws:iam::123456789012:role/SageMakerExecutionRole",
instance_type="ml.m5.large",
instance_count=1,
sagemaker_session=sess,
)
with patch("sagemaker.core.s3.S3Uploader.upload_string_as_file_body"):
processor._package_code(
entry_point="script.py",
source_dir=SOURCE_DIR,
requirements=None,
job_name="test-job",
kms_key=None,
)
```
4. Observe `PermissionError: [WinError 32]`.
**Real-world usage** (also fails before the processing job is created):
```python
from sagemaker.core.helper.session_helper import Session
from sagemaker.core.image_uris import get_training_image_uri
from sagemaker.core.processing import FrameworkProcessor
sess = Session()
processor = FrameworkProcessor(
image_uri=get_training_image_uri(
region=sess.boto_region_name,
framework="pytorch",
instance_type="ml.m5.large",
),
role="",
instance_type="ml.m5.large",
instance_count=1,
)
processor.run(
code="script.py", # script to execute
source_dir="src", # local directory
job_name="test-job",
wait=False,
)
```
**Expected behavior**
Code packaging completes successfully on Windows: the temporary `sourcedir.tar.gz` is removed after upload (or left for the OS to clean up), and `FrameworkProcessor.run()` proceeds to create the processing job.
**Screenshots or logs**
```
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
'C:\\Users\\LORENZ~1\\AppData\\Local\\Temp\\tmp6qp3s0lg.tar.gz'
```
Traceback (abbreviated):
```
File ".../sagemaker/core/processing.py", line 1231, in run
s3_runproc_sh, inputs, job_name = self._pack_and_upload_code(...)
File ".../sagemaker/core/processing.py", line 1270, in _pack_and_upload_code
s3_payload = self._package_code(...)
File ".../sagemaker/core/processing.py", line 1182, in _package_code
os.unlink(tmp.name)
PermissionError: [WinError 32] ...
```
**System information**
- **SageMaker Python SDK version**: `sagemaker-core==2.11.0` (V3 imports: `sagemaker.core.*`)
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: PyTorch (`FrameworkProcessor` with PyTorch training image URI)
- **Framework version**: default from `get_training_image_uri` (py312 image selected automatically)
- **Python version**: 3.14.4
- **CPU or GPU**: CPU (local client); instance type `ml.m5.large` for the job definition
- **Custom Docker image (Y/N)**: N
**Additional context**
**Suggested fix:** Close the temp file before `unlink`, or use `tempfile.mkstemp` and avoid holding an open handle during deletion. For example:
```python
fd, tmp_path = tempfile.mkstemp(suffix=".tar.gz")
os.close(fd)
try:
with tarfile.open(tmp_path, "w:gz") as tar:
...
with open(tmp_path, "rb") as f:
body = f.read()
s3.S3Uploader.upload_string_as_file_body(body=body, ...)
return s3_uri
finally:
if os.path.exists(tmp_path):
os.unlink(tmp_path)
```
Alternatively, move `os.unlink(tmp.name)` **outside** the `NamedTemporaryFile` context manager (after the `with` block exits and the handle is closed).
Contributor guide
Assessment
This issue has not been assessed yet.