Eclectic-Sheep / Eclectic-Sheep/sheeprl

Potential Misalignment in p2e_dv2 and p2e_dv3 Implementations with Original Paper

Open
#322 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
439
Forks
66
PR merge metrics
No merged PRs in 30d

Description

I've noticed a potential misalignment in the p2e_dv2 and p2e_dv3 implementations regarding what the ensemble predicts. According to the Plan2Explore paper, the ensemble should predict the image embedding, not the posterior state. The implementation in `p2e_dv1 `appears aligned with this:

`loss -= next_obs_embedding_dist.log_prob(embedded_obs.detach()[1:]).mean()`

However, in `p2e_dv2 `and `p2e_dv3`, it seems to aim to predict the next (randomized) posterior state:

`loss -= next_obs_embedding_dist.log_prob(posteriors.view(sequence_length, batch_size, -1).detach()[1:]).mean()`

Could this be an intentional modification, or am I missing something about how these predictions should be handled?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by comparing the p2e_dv1, p2e_dv2, and p2e_dv3 ensemble-loss calculations shown in the issue with the Plan2Explore paper. Determine whether the dv2 and dv3 targets should be image embeddings or posterior states, and check for related tests or entry points in those implementations. Done means the intended target is established and the implementations consistently follow it.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.