facebookresearch / facebookresearch/fairscale
containers:apply_to_tensors fails to return (or test) the application result on PackedSequence
Open
FSDP
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 293
- PR merge metrics
- No merged PRs in 30d
Description
At this point in `apply_to_tensors()`, the PackedSequence case drops the result tensors, unlike the other cases
https://github.com/facebookresearch/fairscale/blob/main/fairscale/utils/containers.py#L27
and thus fully_sharded_data_parallel is going to fail to capture the tensors for hooks here:
https://github.com/facebookresearch/fairscale/blob/main/fairscale/nn/data_parallel/fully_sharded_data_parallel.py#L1545
or properly yield casting results here:
https://github.com/facebookresearch/fairscale/blob/main/fairscale/nn/data_parallel/fully_sharded_data_parallel.py#L2490
Contributor guide
Assessment
This issue has not been assessed yet.