Beam search decoding during inference doesn't generate good text.

Hi,

I have trained a model using Reinforcement learning.
When I use **"beam search"** to generate text, it generates all
```
"raeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraera"
```

However, when I use **greedy** or **topk sampling** the generation is like:
```
Sam was watching a movie. He was very focused on the action. He fell asleep. Sam's glasses fell off his face <|endoftext|>eraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraeraera
```
I used the ```tx.utils.strip_eos``` to strip anything after <|endoftext|>.

1- I am not sure why beam search is performing this way? I would appreciate your help. following is my piece of code for doing decoding using beam search:
```
def _infer_beam_ids(context_name):
# beam-search
predictions = decoder(
beam_width=10,
length_penalty=config_train.length_penalty,
embedding=_embedding_fn,
context=batch['%s_ids' % context_name],
context_sequence_length=batch['%s_len' % context_name],
max_decoding_length=max_decoding_length,
end_token=end_token,
mode=tf.estimator.ModeKeys.PREDICT)

beam_output_ids = tx.utils.varlength_roll(predictions["sample_id"][:, :, 0], -batch['%s_len' % context_name], axis=1)

return beam_output_ids

beam_search_ids = _infer_beam_ids('x1')
```

2- Is it better to use beam search for a model which is trained in a _self-critical fashion_, right?

I would appreciate if you can help me with these.

Start with the _infer_beam_ids entry point and inspect the decoder call, tx.utils.varlength_roll, and tx.utils.strip_eos usage. Compare beam-search output with greedy and top-k sampling, then document the cause of the repeated text and whether self-critical training changes the appropriate decoding method.

Written by the indexing model from the issue text.