microsoft / microsoft/LatentSpatialMemory

Question regarding scene transitions/cuts in processed RealEstate10K clips for ViPE

Open
#15 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
325
Forks
11
Avg merge
1h 16m
Merged PRs (30d)
3

Description

Hello,

Thanks a lot for your great work!
I am attempting to reproduce the data pipeline by running python -m data_process.run_video_collect on the raw RealEstate10K videos to generate clip.mp4 samples and subsequently ran ViPE pose estimation on them.

In several generated clips, I noticed subtle scene transitions/cuts in the middle of the video.

https://github.com/user-attachments/assets/d8056aa1-cccd-46ad-aae3-0362aa61e15a

Because ViPE relies on sequential frames, these cuts lead to incorrect pose estimates being assigned to the frames following the transition.

Are there additional post-processing filtering steps used to prune these non-sequential clips, or was this not encountered during video collection?

Thanks!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the data_process.run_video_collect entry point and reproduce the reported clip.mp4 generation from raw RealEstate10K videos. Inspect whether the collection pipeline includes post-processing that detects or removes scene transitions, then compare the resulting frame sequence with ViPE pose estimates. Done means determining whether such filtering exists and documenting or addressing the behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
computer-vision, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.