actions/logged seem mismatched in TensorBoard

Hi guys,

I've been using Horizon to train a discrete_action DQN. My state features are labeled by "0","1","2", and there are 100 possible discrete actions. In the training timeline data, my actions are named 0,1,2,...,99. I trained the data using:

python ml/rl/workflow/dqn_workflow.py -p dqn.json

Since the state features has taken up 0,1,and 2, in dqn.json I listed my actions as "3", "4", ..., "102". I have also attached my simplified dqn.json file here.
[dqn.txt](https://github.com/facebookresearch/Horizon/files/3347022/dqn.txt)

After the training is done, I then visualize the results with TensorBoard. And in the "actions" section I see something like this:

I have several questions about this:

1. Does actions/logged/4 describes how many times the action "4" shows up in each epoch? If so, why is the number oscillating? I would imagine that each epoch goes through the same training data and therefore the action count should be a constant?

2. In my training set, action 0 did not happen at all. I would imagine in Horizon's case that means action "3" should have a count of 0 (I assume the action correspondence is 0-"3", 1-"4",...,99-"102"). However, that is not the case as shown below:

My training set also does not have any actions beyond 60, which I interpret as I should see a count of 0 for all actions beyond "63" in TensorBoard, but instead I see count of 0 for actions between "61" and "101", but then oscillations around 8000 in "102".

Is there some other rules for corresponding the actions in training data and in dqn.json? I have been reading the code, but could not find where Horizon corresponds the training data action names to the action names defined in dqn.json. It would be great if you can point me to the part of the code that takes care of this correspondence.

**Update:** I have taken a closer look at action count in my training timeline data, and I am pretty sure the action correspondence between my action names (0,...,99) and dqn.json action names ("3",...,"102") is as follows:
3-"3",...,99-"99", 1 **and** 2 - "102", and then I am not sure if 0 corresponds to "100" or "101" since they all have 0 count. This is really weird. Do you guys have any idea how this comes to be?

I also noticed that some actions have slightly lower count in TensorBoard than in the training timeline data (e.g. 34 vs 35, 239 vs 241). Is it because the training data size cannot be perfectly divided by minibatch_size so some samples are not included in training?

Thank you!

Best,
Fengdan