aws / aws/sagemaker-python-sdk

Add configuration of loggers for PySpark

未关闭
#4,082 1 条评论 1 个 reaction 已指派 1 人 已被 @nargokul 认领 在 GitHub 查看
component: processing type: logging/error reporting
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**Describe the feature you'd like**
Currently, everything for PySpark (Processor, training etc.) is logged at INFO level, including basic setup of cluster, for example:
```
yarn.resourcemanager.hostname 10.0.215.164 The hostname of the RM. yarn.nodemanager.hostname algo-1 The hostname of the NM. yarn.nodemanager.webapp.address algo-1:8042 yarn.nodemanager.vmem-pmem-ratio 5 Ratio between virtual memory to physical memory. yarn.resourcemanager.am.max-attempts 1 The maximum number of application attempts. yarn.nodemanager.env-whitelist JAVA_HOME,HADOOP_COMMON_HOME,HADOOP_HDFS_HOME,HADOOP_CONF_DIR,YARN_HOME,AWS_CONTAINER_CREDENTIALS_RELATIVE_URI,AWS_REGION Environment variable whitelist yarn.scheduler.minimum-allocation-mb 1 yarn.scheduler.maximum-allocation-mb 32768 yarn.scheduler.minimum-allocation-vcores 1 yarn.scheduler.maximum-allocation-vcores 8 yarn.nodemanager.resource.memory-mb 32768 yarn.nodemanager.resource.cpu-vcores 8
--
 



```
This results in a lot of CloudWatch logs, with major downsides:
- most of the logs are completely useless, printing internal details of infrastructure
- searching logs for anything actually important is very hard, which is very problematic for training and monitoring mission-critical models
- it is quite costly for running short jobs, making CloudWatch costs high compared to actual compute costs

Adding options to configure log4j logger, or at least some options to limit this (e.g. minimal logging level), would be to get rid of this. It is also very simple to implement.

**How would this feature be used? Please describe.**
Additional argument(s) passed to e.g. `PySparkProcessor`.

**Describe alternatives you've considered**
Full customizability is not necessarily required, but setting minimal log level is very important.

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。