google / google/dopamine

Use dopamine as a text classifier

Open
#89 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
10.9k
Forks
1.4k
PR merge metrics
No merged PRs in 30d

Description

Hello,
I work on AI project. I need to create with gym an environment and test it on dopamine with his agent (rainbow, dqn, ...). I already created this environment but cant launch him with Dopamine:

My environment is a text classifier. The AI read a sentence and them she can choose one language in list, so she need to detect language in the current sentence. Every step, she read another sentence and give an answer ect...
I give to AI a string value with no static length.

I observed the c51_cartpole.gin and gym_lib.py to try to understand how can add my environment to dopamine project.

**Language is name of my model**

"gym_lib":
CARTPOLE_MIN_VALS = np.array([-2.4, -5., -math.pi/12., -math.pi*2.])
CARTPOLE_MAX_VALS = np.array([2.4, 5., math.pi/12., math.pi*2.])
LANGUAGE_MIN_VALS = ""
LANGUAGE_MAX_VALS = ""
ACROBOT_MIN_VALS = np.array([-1., -1., -1., -1., -5., -5.])
ACROBOT_MAX_VALS = np.array([1., 1., 1., 1., 5., 5.])
gin.constant('gym_lib.CARTPOLE_OBSERVATION_SHAPE', (4, 1))
gin.constant('gym_lib.CARTPOLE_OBSERVATION_DTYPE', tf.float32)
gin.constant('gym_lib.CARTPOLE_STACK_SIZE', 1)
gin.constant('gym_lib.ACROBOT_OBSERVATION_SHAPE', (6, 1))
gin.constant('gym_lib.ACROBOT_OBSERVATION_DTYPE', tf.float32)
gin.constant('gym_lib.ACROBOT_STACK_SIZE', 1)
gin.constant('gym_lib.LANGUAGE_OBSERVATION_SHAPE', (1, 1))
gin.constant('gym_lib.LANGUAGE_OBSERVATION_DTYPE', tf.string)
gin.constant('gym_lib.LANGUAGE_STACK_SIZE', 51)

I try to follow the current organisation but i think i have problem with entry data.

"c51_language.gin"
# Hyperparameters for a simple C51-style Language agent. The hyperparameters
# chosen achieve reasonable performance.
import dopamine.agents.dqn.dqn_agent
import dopamine.agents.rainbow.rainbow_agent
import dopamine.discrete_domains.gym_lib
import dopamine.discrete_domains.run_experiment
import dopamine.replay_memory.prioritized_replay_buffer
import gin.tf.external_configurables

RainbowAgent.observation_shape = %gym_lib.LANGUAGE_OBSERVATION_SHAPE
RainbowAgent.observation_dtype = %gym_lib.LANGUAGE_OBSERVATION_DTYPE
RainbowAgent.stack_size = %gym_lib.LANGUAGE_STACK_SIZE
RainbowAgent.network = @gym_lib.language_rainbow_network
RainbowAgent.num_atoms = 51
RainbowAgent.vmax = 10.
RainbowAgent.gamma = 0.99
RainbowAgent.update_horizon = 1
RainbowAgent.min_replay_history = 500
RainbowAgent.update_period = 4
RainbowAgent.target_update_period = 100
RainbowAgent.epsilon_fn = @dqn_agent.identity_epsilon
RainbowAgent.replay_scheme = 'uniform'
RainbowAgent.tf_device = '/gpu:0' # use '/cpu:*' for non-GPU version
RainbowAgent.optimizer = @tf.train.AdamOptimizer()

tf.train.AdamOptimizer.learning_rate = 0.001
tf.train.AdamOptimizer.epsilon = 0.0003125

create_gym_environment.environment_name = 'language'
create_gym_environment.version = 'v0'
create_agent.agent_name = 'rainbow'
Runner.create_environment_fn = @gym_lib.create_gym_environment
Runner.num_iterations = 500
Runner.training_steps = 1000
Runner.evaluation_steps = 1000
Runner.max_steps_per_episode = 200 # Default max episode length.

WrappedPrioritizedReplayBuffer.replay_capacity = 50000
WrappedPrioritizedReplayBuffer.batch_size = 128

Is it possible to add gym environment with string entry in dopamine ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.