aws / aws/sagemaker-python-sdk

`metric_definitions` documentation claims arbitrary regex but requires metrics to be integers.

Abierto
#3,208 0 comentarios 0 reacciones 1 asignado Reclamado por @nargokul Ver en GitHub
type: bug
Lenguaje dominante
Python
Estrellas
2.3k
Forks
1.3k
Merge medio
1 d 22 h
PR fusionados (30 d)
35

Descripción

**Describe the bug**
The documentation for [SKLearn](https://sagemaker.readthedocs.io/en/stable/frameworks/sklearn/sagemaker.sklearn.html?highlight=SKLearn#sagemaker.sklearn.estimator.SKLearn) constructor mentions that we can use custom `metric_definitions` with a regex. I checked that this kwarg is passed to `Framework` which then passes it to [EstimatorBase](https://sagemaker.readthedocs.io/en/stable/api/training/estimators.html#sagemaker.estimator.EstimatorBase).

This regex only captures numbers and not strings. This should either be mentioned in the documentation or support fort arbitrary strings needs to be present.

**To reproduce**

`cli.py`
```python
if __name__ == "__main__":
print("canary=1.01;")
print(f"cat=dog;")
```

`sagemaker.ipynb`
```python
from sagemaker.sklearn.estimator import SKLearn
import sagemaker
from sagemaker import get_execution_role
from copy import deepcopy
import itertools

sagemaker_session = sagemaker.Session()
role = get_execution_role()

FRAMEWORK_VERSION = "1.0-1"
script_path = "cli.py"

LOCAL=False

kwargs=dict(
instance_type="ml.m5.large",
instance_count=1,
sagemaker_session=sagemaker_session,
base_job_name="test-job-",
)

base_model_gen = lambda: EstimatorBase(
entry_point=script_path,
framework_version=FRAMEWORK_VERSION,
role=role,
source_dir="./",
metric_definitions=[dict(Name=m, Regex=f"{m}=(.*?);") for m in ["canary",'cat']],
**kwargs
)
base_model_gen().fit()
```

**Expected behavior**
Algorithm metrics should've displayed both `canary` and `cat`
Only `canary` is displayed

**Screenshots or logs**
![image](https://user-images.githubusercontent.com/108535060/176913863-29953eb1-6d04-4ce5-a43c-b61d50367679.png)

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: '2.97.0'
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: SKLearn
- **Framework version**: 1.0-1
- **Python version**: 3.8.12
- **CPU or GPU**: CPU
- **Custom Docker image (Y/N)**: N

**Additional context**
Add any other context about the problem here.

Guía de contribución

Abrir la guía de contribución

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.