microsoft / microsoft/LatentSpatialMemory
Question regarding scene transitions/cuts in processed RealEstate10K clips for ViPE
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 325
- Forks
- 11
- Avg merge
- 1h 16m
- Merged PRs (30d)
- 3
Description
Hello,
Thanks a lot for your great work!
I am attempting to reproduce the data pipeline by running python -m data_process.run_video_collect on the raw RealEstate10K videos to generate clip.mp4 samples and subsequently ran ViPE pose estimation on them.
In several generated clips, I noticed subtle scene transitions/cuts in the middle of the video.
https://github.com/user-attachments/assets/d8056aa1-cccd-46ad-aae3-0362aa61e15a
Because ViPE relies on sequential frames, these cuts lead to incorrect pose estimates being assigned to the frames following the transition.
Are there additional post-processing filtering steps used to prune these non-sequential clips, or was this not encountered during video collection?
Thanks!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the data_process.run_video_collect entry point and reproduce the reported clip.mp4 generation from raw RealEstate10K videos. Inspect whether the collection pipeline includes post-processing that detects or removes scene transitions, then compare the resulting frame sequence with ViPE pose estimates. Done means determining whether such filtering exists and documenting or addressing the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- computer-vision, data-engineering
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100