google-deepmind / google-deepmind/pysc2
I have some questions about pysc2 baseline agent of Deepmind.
- Dominant language
- Python
- Stars
- 8.3k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
I have some questions about pysc2 baseline agent of Deepmind.
https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/
I'm trying to implement baseline agent illustrated in the pysc2 paper.
My GitHub: https://github.com/chris-chris/pysc2-examples
Question1. How can I implement 'the policy in an auto-regressive manner, utilizing the chain rule'?
1. I got the idea of masking unavailable actions from policy network result.
2. But I don't get the idea of 'the policy in an auto-regressive manner, utilizing the chain rule'
If I choose the first action, then how can I get the second parameters and the third action parameters in an auto-regressive manner?

Question2. How can I define 'action space' for A3C for pysc2? (Similar one with Q1)
I'm kind of familiar with Discrete action space.
1. I got the concept of calculating the spatial action policy using 1x1 convolution layer for dimension reduction.
2. But what the action space would look like?
It should not be like tf.int32.
Maybe it can be MultiDiscrete action space.
[move_screen(categorical int), [[0](categorical int), (x1(int), y1(int))]]
3. If it is a MultiDiscrete Action Space, then how can I define policy loss function?

If you know some reference codes on Github, please let me know.
or some relevant papers and articles would be welcome.
Contributor guide
Assessment
This issue has not been assessed yet.