facebookresearch / facebookresearch/metaquery
Clarification on the 1,000-step image-editing fine-tuning recipe
- Dominant language
- Python
- Stars
- 327
- Forks
- 14
- PR merge metrics
- No merged PRs in 30d
Description
Hi, thank you for releasing MetaQuery.
I am trying to understand the exact image-editing protocol described in Section 5.3, where the paper reports fine-tuning the pretrained MetaQuery-B model for 1,000 steps on publicly available editing data.
At commit `a44e6e20edf25469a8160a244411950f5e4dcd25`, I found:
- [`llavaov0p5_sana_edit.yaml`](https://github.com/facebookresearch/metaquery/blob/a44e6e20edf25469a8160a244411950f5e4dcd25/configs/llavaov0p5_sana_edit.yaml) specifies:
- `ominiedit: -1`
- per-device batch size 32
- 5 epochs
- 5,000 warmup steps
- no `max_steps: 1000`
- [`dataset.py`](https://github.com/facebookresearch/metaquery/blob/a44e6e20edf25469a8160a244411950f5e4dcd25/dataset.py#L302-L331) maps `ominiedit` to `TIGER-Lab/OmniEdit-Filtered-1.2M`.
- [`train.py`](https://github.com/facebookresearch/metaquery/blob/a44e6e20edf25469a8160a244411950f5e4dcd25/train.py#L144-L160) loads an externally supplied `--resume_from_checkpoint`, but the README does not identify the checkpoint used in the paper.
Assuming 8 GPUs, batch size 32/GPU, and gradient accumulation 1, five epochs over 1.2M samples would be approximately 23.4K optimizer updates rather than 1,000.
Could you please clarify:
1. Was the paper’s 1,000-step editing run trained on `TIGER-Lab/OmniEdit-Filtered-1.2M`? If not, which dataset/version was used?
2. Does 1,000 steps mean optimizer updates, and what was the effective global batch size?
3. What exact starting checkpoint was used? Was it already fine-tuned for image reconstruction?
4. What LR, warmup, scheduler, and seed were used? Was the released 5,000-step warmup used?
5. Were MetaQueries, connector, and the full diffusion transformer trained?
6. What source-image/text dropout and inference CFG settings were used?
7. Is the exact 1,000-step config or starting checkpoint available?
I am asking because I am adapting a related MetaQuery-style architecture for source-image instruction editing. The checkpoint stage, effective sample count, and conditioning dropout materially affect both compute cost and the risk of the model ignoring the source image.
Thanks!
Contributor guide
Research direction
Review configs/llavaov0p5_sana_edit.yaml, dataset.py, train.py, and the README alongside Section 5.3. Determine whether the released configuration corresponds to the paper's 1,000-step run and document the dataset, checkpoint, optimization settings, trained components, dropout, and CFG details. Done means the README or an exact config answers the listed reproducibility questions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100