RuntimeError: result type Float can't be cast to desired output type Long

Hello!

I'm trying to run the `hierar-or-linear` experiment described in the paper on transformers with the following settings:
```
python local_grid.py --sweep=hyperparams/hierar-or-linear/transformer_small.json --task=tasks/hierar-or-linear/4/fpa/ --n_workers=1
```

However, I'm getting a RuntimeError at the following line in `generate.py`:
```python
hypos = task.inference_step(generator, models, sample, prefix_tokens)
```
Note that in order to see this error message, I had to tweak the script to turn off parallelization; otherwise, it failed silently and I didn't see the message appear in the `stderr` file (or `stdout`) in the results directory (I originally noticed what was going on because I didn't see the generation results in `stdout`).

Here's the full error message:
```
Traceback (most recent call last):
File "local_grid.py", line 125, in
combined_run(train_params)
File "local_grid.py", line 76, in combined_run
generate_main(generate_params)
File "/gpfs/gibbs/project/frank/maw244/FIND/generate.py", line 98, in cli_main
main(args)
File "/gpfs/gibbs/project/frank/maw244/FIND/generate.py", line 66, in main
hypos = task.inference_step(generator, models, sample, prefix_tokens)
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/fairseq/tasks/fairseq_task.py", line 265, in inference_step
return generator.generate(models, sample, prefix_tokens=prefix_tokens)
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/fairseq/sequence_generator.py", line 113, in generate
return self._generate(model, sample, **kwargs)
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/fairseq/sequence_generator.py", line 376, in _generate
cand_scores, cand_indices, cand_beams = self.search.step(
File "/gpfs/gibbs/project/frank/maw244/conda_envs/inductive/lib/python3.8/site-packages/fairseq/search.py", line 81, in step
torch.div(self.indices_buf, vocab_size, out=self.beams_buf)
RuntimeError: result type Float can't be cast to the desired output type Long
```
I took a look at the dtypes of the arguments to `task.inference_step`, but all the tensors are already type `torch.int64`, and I'm not sure where the float type originates internally.

Here are the versions of the packages in the installation instructions that I'm using:
```
python 3.8.16
pytorch 2.0.0, build py3.8_cuda11.8_cudnn8.7.0_0
torchvision 0.15.0, build py38_cud118
cudatoolkit 11.8.0, build h37601d7_11
fairseq 0.9.0, build py38hbf85e49_0
```
One minor thing I had to change when setting up the environment is I had to use conda to install fairseq; using pip produced an error that it wasn't able to build the wheel correctly, but conda finished fine. Not sure if that's relevant. Happy to provide any more info that might help identify what the source of the issue is!