CPE examples as in article

Hello, I am trying to include CPE metrics into Tensorboard report as it was done on the Figure 2 in this paper. Also, I'd like to depict chart with difference doubly robust and others estimators.
I suppose that I need to add some metrics, could you please advice, where can I do this?
What does it mean logged_metrics in EvaluationDataPage, how to initialize them?
It will be wonderful in case metrics_to_score can be explained.
https://arxiv.org/pdf/1811.00260.pdf

I added parameter, but it didn't help.
evaluation:
calc_cpe_in_training: true
I debugged and noticed, that ExtraData metrics field is populated with None value. I am using DiscreteDqn.
Could you please advice me how it should look like? Where should I include some additional configurations or something?