Eclectic-Sheep / Eclectic-Sheep/sheeprl

Question about the `compute_lambda_values` function

Open
#302 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
439
Forks
66
PR merge metrics
No merged PRs in 30d

Description

Hi! Thanks for your work on reimplementing `dreamverv1` in a simple way.

I tried to learn the computation process of `dreamerv1`, but feel confused about the logistics of the `compute_lambda_values` function:

https://github.com/Eclectic-Sheep/sheeprl/blob/dee8c8010bdff9bc3fd81669ffd4c90513f9b578/sheeprl/algos/dreamer_v1/utils.py#L66-L77

1. Does the above snippet refer to Eq.6 in the original paper? i.e.,

```math
V_\lambda(s_\lambda) = (1- \lambda) \sum_{n-1}^{H-1} \lambda ^{n-1} V_N^n(s_\lambda) + \lambda ^{H-1} V_N^H(s_\lambda)
```
I could not find anything in common between them.

2. If so, what does `delta` mean? Is `delta` TD target?

I'm new to the Dreamer series. Please forgive me if my question looks dumb to you. Thanks.

Contributor guide

No contributing guide indexed for this repository

Research direction

Read sheeprl/algos/dreamer_v1/utils.py at lines 66-77 alongside Eq. 6 of the original Dreamer v1 paper. Trace how compute_lambda_values uses delta and identify whether the implementation corresponds to the paper's notation. Done means the relationship between the snippet, Eq. 6, and delta is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.