baidut / baidut/PatchVQ

Cannot reproduce the PVQ numbers in test_PVQ_on_new_datasets.ipynb

Open
#10 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
76
Forks
13
PR merge metrics
No merged PRs in 30d

Description

Hi,

Very interesting work!

So for fun I tried following every step in `test_PVQ_on_new_datasets.ipynb` (the only difference being that I'm running a bunch of smaller Python programs instead a Jupyter notebook), but I'm getting different values on `LIVE_VQC_tiny_all`:

> PVQ 0.704,0.605

Instead of 0.548,0.515 as stated in `test_PVQ_on_new_datasets.ipynb`. Are the numbers in the Jupyter notebook outdated perhaps?

I verified every intermediate step and everything seems to be OK in terms of tensor sizes and so on.

I'm curious if anyone has been able to reproduce the stated PVQ numbers successfully from the Jupyter notebook.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by rerunning the steps in test_PVQ_on_new_datasets.ipynb for LIVE_VQC_tiny_all and compare the reported PVQ values with 0.704,0.605 and 0.548,0.515. Check whether the notebook's values are outdated and use the reported intermediate tensor-size checks to narrow down the discrepancy. Done means identifying the source of the differing results and documenting or correcting the reproducible values.

Written by the indexing model from the issue text.

Assessment

Tech stack
jupyter-notebook, python
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.