awslabs / awslabs/sagemaker-debugger
Tensors not saved in PREDICT step
- Dominant language
- Python
- Stars
- 165
- Forks
- 82
- PR merge metrics
- No merged PRs in 30d
Description
```
if self._prepared_tensors[mode]:
if self._exported_collections is False:
# in keras, these collections change when mode changes
# but rest of the project isn't yet capable of handling this
# this means that collections like outputs, or other collections with intermediate tensors
# will only have tensor names from first mode
# this means sometimes collections will be exported after 1 step
self.export_collections()
self._exported_collections = True
```
If we run the same mode consecutively, we end up exporting collections before all the modes are run.
Contributor guide
Assessment
This issue has not been assessed yet.