Eclectic-Sheep / Eclectic-Sheep/sheeprl
Potential Misalignment in p2e_dv2 and p2e_dv3 Implementations with Original Paper
- Dominant language
- Python
- Stars
- 439
- Forks
- 66
- PR merge metrics
- No merged PRs in 30d
Description
I've noticed a potential misalignment in the p2e_dv2 and p2e_dv3 implementations regarding what the ensemble predicts. According to the Plan2Explore paper, the ensemble should predict the image embedding, not the posterior state. The implementation in `p2e_dv1 `appears aligned with this:
`loss -= next_obs_embedding_dist.log_prob(embedded_obs.detach()[1:]).mean()`
However, in `p2e_dv2 `and `p2e_dv3`, it seems to aim to predict the next (randomized) posterior state:
`loss -= next_obs_embedding_dist.log_prob(posteriors.view(sequence_length, batch_size, -1).detach()[1:]).mean()`
Could this be an intentional modification, or am I missing something about how these predictions should be handled?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by comparing the p2e_dv1, p2e_dv2, and p2e_dv3 ensemble-loss calculations shown in the issue with the Plan2Explore paper. Determine whether the dv2 and dv3 targets should be image embeddings or posterior states, and check for related tests or entry points in those implementations. Done means the intended target is established and the implementations consistently follow it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100