Eclectic-Sheep / Eclectic-Sheep/sheeprl
Question about the `compute_lambda_values` function
- Dominant language
- Python
- Stars
- 439
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
Hi! Thanks for your work on reimplementing `dreamverv1` in a simple way.
I tried to learn the computation process of `dreamerv1`, but feel confused about the logistics of the `compute_lambda_values` function:
https://github.com/Eclectic-Sheep/sheeprl/blob/dee8c8010bdff9bc3fd81669ffd4c90513f9b578/sheeprl/algos/dreamer_v1/utils.py#L66-L77
1. Does the above snippet refer to Eq.6 in the original paper? i.e.,
```math
V_\lambda(s_\lambda) = (1- \lambda) \sum_{n-1}^{H-1} \lambda ^{n-1} V_N^n(s_\lambda) + \lambda ^{H-1} V_N^H(s_\lambda)
```
I could not find anything in common between them.
2. If so, what does `delta` mean? Is `delta` TD target?
I'm new to the Dreamer series. Please forgive me if my question looks dumb to you. Thanks.
Contributor guide
No contributing guide indexed for this repository
Research direction
Read sheeprl/algos/dreamer_v1/utils.py at lines 66-77 alongside Eq. 6 of the original Dreamer v1 paper. Trace how compute_lambda_values uses delta and identify whether the implementation corresponds to the paper's notation. Done means the relationship between the snippet, Eq. 6, and delta is clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100