google-deepmind / google-deepmind/pysc2

I have some questions about pysc2 baseline agent of Deepmind.

Open
#88 8 comments 0 reactions 1 assignee Claimed by @OriolVinyals View on GitHub
agent
Dominant language
Python
Stars
8.3k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

I have some questions about pysc2 baseline agent of Deepmind.

https://deepmind.com/blog/deepmind-and-blizzard-open-starcraft-ii-ai-research-environment/

I'm trying to implement baseline agent illustrated in the pysc2 paper.

My GitHub: https://github.com/chris-chris/pysc2-examples

Question1. How can I implement 'the policy in an auto-regressive manner, utilizing the chain rule'?
1. I got the idea of masking unavailable actions from policy network result.
2. But I don't get the idea of 'the policy in an auto-regressive manner, utilizing the chain rule'
If I choose the first action, then how can I get the second parameters and the third action parameters in an auto-regressive manner?

2017-10-07 5 42 47

Question2. How can I define 'action space' for A3C for pysc2? (Similar one with Q1)
I'm kind of familiar with Discrete action space.
1. I got the concept of calculating the spatial action policy using 1x1 convolution layer for dimension reduction.
2. But what the action space would look like?
It should not be like tf.int32.
Maybe it can be MultiDiscrete action space.
[move_screen(categorical int), [[0](categorical int), (x1(int), y1(int))]]
3. If it is a MultiDiscrete Action Space, then how can I define policy loss function?

2017-10-07 6 10 06

If you know some reference codes on Github, please let me know.
or some relevant papers and articles would be welcome.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.