awslabs / awslabs/sagemaker-debugger

[Feature Request] TensorBoardOutputConfig local output path

Open
#560 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
165
Forks
82
PR merge metrics
No merged PRs in 30d

Description

Hi, I really wish I could use `TensorBoardOutputConfig` when my SageMaker Estimator instance type is `local` or `local_gpu`.

This could be accomplished by either or both of:
- Enabling streaming to s3 when running local mode jobs
- Enabling a local output path instead of an s3 output path, for example:

```python
from sagemaker.debugger import TensorBoardOutputConfig

instance_type = 'local_gpu'

tb_output_path = "file:///local/path/to/stream/tensorboard/files/to"

tensorboard_output_config = TensorBoardOutputConfig(
# and rename s3_output_path to output_path?
output_path=tb_output_path,
container_local_output_path='/tensorboard'
)
```
This would make it possible to perform local debugging of tensorboard logging code within my sagemaker scripts. Without local debugging, writing tensorboard logging code for sagemaker jobs can be a very slow process.

Contributor guide

Open the contributing guide

Research direction

Start with TensorBoardOutputConfig and the local and local_gpu Estimator modes described in the issue. Determine whether the change should support streaming to S3, a file:// output path, or both, then verify that local debugging of TensorBoard logging works as requested.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.