Question on Chapter 18 - loss functions
- Dominant language
- Jupyter Notebook
- Stars
- 30k
- Forks
- 13.1k
- PR merge metrics
- No merged PRs in 30d
Description
Greetings, I'm working through the cartpole example on page 695 of the third edition, and I have a question about the code presented:
```
def play_one_step(env, obs, model, loss_fn):
with tf.GradientTape() as tape:
left_proba = model(obs[np.newaxis])
action = (tf.random.uniform([1, 1]) > left_proba)
y_target = tf.constant([[1.]]) - tf.cast(action, tf.float32)
loss = tf.reduce_mean(loss_fn(y_target, left_proba))
grads = tape.gradient(loss, model.trainable_variables)
obs, reward, done, truncated, info = env.step(int(action))
return obs, reward, done, truncated, grads
```
I'm confused about y_target, and why it's an input into the loss function. If the action is False (0), y_target is 1. If the action is True (1), y_target is 0. It seems like we are effectively saying that the model should have been more confident in whatever it's output was. Is that the correct way to think about what y_target is accomplishing? If so, is there something happening in a later step where we're determining if the action recommended by the model was beneficial?
I have similar questions about the loss function presented on page 710, but if I can get some clarification on this earlier example, perhaps I'll understand the more challenging Q-value example.
Thank you!
Contributor guide
No contributing guide indexed for this repository
Research direction
Read the CartPole example in Chapter 18, page 695, including play_one_step and its loss function, then compare it with the Q-value loss discussion on page 710. Done means providing a clear explanation of y_target, how the loss is used, and where the example determines whether an action was beneficial.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python, tensorflow
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100