Probabilistic actors during evaluation

Hello,

How does evaluation_deterministic_actions in the experiment config affect the agents' behaviour during evaluation phase?

1. My understanding of policy-based algorithms is that they learn a policy, and they sample actions from the policy. How can they deterministically choose an action from a random distribution? Do they deterministically pick the action which has the highest probability?
2. For value-based algorithms like IQL, if I turn off evaluation_deterministic_actions, does that mean IQL will still use the e-greedy scheme to choose actions at evaluation phase?
3. Is it possible to make IQL behave according to some other randomization scheme other than e-greedy during training and/or evaluation?

Thank you.

The issue names the evaluation_deterministic_actions experiment setting and IQL, but no files or tests. Start by tracing where that setting reaches evaluation action selection and how policy-based and value-based algorithms choose actions. Done means documenting deterministic versus stochastic behavior, IQL's evaluation randomization, and the available training and evaluation schemes.

Written by the indexing model from the issue text.