NCCL issue training with two GPUs

I ran into this issue (https://github.com/NVIDIA/nccl/issues/1125) when trying to replicate the instructions from the README. Since the blog posts mentions that the training was done on two GPUs is there a workaround for the NCCL issue with 1 or 2 GPUs?

Ran
$ python train.py --model_name meta-llama/Llama-2-70b-hf --batch_size 2 --context_length 2048 --precision bf16 --train_type qlora --use_gradient_checkpointing true --use_cpu_offload true --dataset alpaca --reentrant_checkpointing true

The error trace looks like -

torch.distributed.DistBackendError: NCCL error in: ../torch/csrc/distributed/c10d/ProcessGroupNCCL.cpp:1691, internal error - please report this issue to the NCCL developers, NCCL version 2.19.3
ncclInternalError: Internal check failed.
Last error:
Attribute busid of node nic not found

Start by reproducing the README command with one or two GPUs and compare the resulting NCCL error with the linked NCCL issue. Determine whether the reported setup has a documented workaround; done means documenting a verified workaround or clearly recording that the configuration is unsupported.

Written by the indexing model from the issue text.