List of bugs in different games

Hi, I have worked a lot with atari-py (I first tried to report this issue there openai/atari-py#41 but they told me this was likely to be coming from there) recently and discovered some bugs which I think can be damageable for the research. Here is the list I found :

- Asterix : When my agent reach 999500 and take a last bonus which should lead him to 1000000 (the max score), it obtains a reward of -999500 and the game continue as usual (but the agent now got a total score of 0...). I think this issue can be seen on the score reported in Rainbow or Ape-X paper (the score goes up to 1M and then vary randomly around 500k).

- Defender : On this game reward are really weird. I got a reward of 10 the first time step no matter what. And then all reward are multiplied by 100 (so I got a reward 15000 when the actual render of the score on the screen add 150 to the score). Moreover like in Asterix, my agent got a reward of -999000-ish when he got more than 1M score.

- VideoPinball : Same than Asterix and Defender, agent receives a reward of -999000-ish when he reach 1M score.

- BattleZone : Often my agent get stuck for ever for no reason. By stuck I mean than when this kind of state happen, even playing random action for 20 hours doesn't finish the game and the agent never receives reward different than 0. This is for me an issue particularly with algorithm relaying on replay memory, when this happen, the replay memory get filled with tons of useless transitions. I could report the random seed and the list of action leading to one of those state if needed (I am using sticky actions with probability 0.25).

- Yar'sRevenge : Same than for BattleZone, sometimes the game get stuck for ever. But this happen way less often than in BattleZone though...