aws / aws/sagemaker-python-sdk

`metric_definitions` documentation claims arbitrary regex but requires metrics to be integers.

Đang mở
#3,208 0 bình luận 0 reaction 1 người được giao Được @nargokul nhận Xem trên GitHub
type: bug
Ngôn ngữ chính
Python
Star
2.3k
Fork
1.3k
Merge trung bình
1 ngày 22 giờ
Pull request đã merge (30 ngày)
35

Mô tả

**Describe the bug**
The documentation for [SKLearn](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/sagemaker.sklearn.html?highlight=SKLearn#sagemaker.sklearn.estimator.SKLearn) constructor mentions that we can use custom `metric_definitions` with a regex. I checked that this kwarg is passed to `Framework` which then passes it to [EstimatorBase](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.EstimatorBase).

This regex only captures numbers and not strings. This should either be mentioned in the documentation or support fort arbitrary strings needs to be present.

**To reproduce**

`cli.py`
```python
if __name__ == "__main__":
print("canary=1.01;")
print(f"cat=dog;")
```

`sagemaker.ipynb`
```python
from sagemaker.sklearn.estimator import SKLearn
import sagemaker
from sagemaker import get_execution_role
from copy import deepcopy
import itertools

sagemaker_session = sagemaker.Session()
role = get_execution_role()

FRAMEWORK_VERSION = "1.0-1"
script_path = "cli.py"

LOCAL=False

kwargs=dict(
instance_type="ml.m5.large",
instance_count=1,
sagemaker_session=sagemaker_session,
base_job_name="test-job-",
)

base_model_gen = lambda: EstimatorBase(
entry_point=script_path,
framework_version=FRAMEWORK_VERSION,
role=role,
source_dir="./",
metric_definitions=[dict(Name=m, Regex=f"{m}=(.*?);") for m in ["canary",'cat']],
**kwargs
)
base_model_gen().fit()
```

**Expected behavior**
Algorithm metrics should've displayed both `canary` and `cat`
Only `canary` is displayed

**Screenshots or logs**
![image](https://user-images.githubusercontent.com/108535060/176913863-29953eb1-6d04-4ce5-a43c-b61d50367679.png)

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: '2.97.0'
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: SKLearn
- **Framework version**: 1.0-1
- **Python version**: 3.8.12
- **CPU or GPU**: CPU
- **Custom Docker image (Y/N)**: N

**Additional context**
Add any other context about the problem here.

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Đánh giá

Issue này chưa được đánh giá.

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.