Finetune on Spring dataset questions
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 3k
- Forks
- 234
- PR merge metrics
- No merged PRs in 30d
Description
Hello, thank you for releasing the training code. :)
I have a few questions. I want to finetune the model on an existing dataset that you noted in your supplementary materials, on the spring dataset. So I did a comparison with the stock version of the model and I obtained some metrics.
Note: The "test" spring dataset was obtained from the hugging face preprocessed datasets you mentioned in the eval section.
After that I took the entire train Spring dataset, and crafted the structure required to finetune the model on the Spring dataset, I followed your advice on the learning rates being 1e-5 for the decoder and 1e-6 for the encoder, and finetuned the model for an exact 1000 iterations ( too low?, it takes a while for a bigger number of iterations )
After doing so, I again reevaluated the new model with the new weights and observed my results are now slightly deviated from the original weights evaluation, being a little worse.
Questions:
Regarding the dataset crafting, in the meta.json structure, I also rescalled the intrinsics from:
{
"intrinsics": [
[
2181.8181818181815,
0,
960.0
],
[
0,
2181.8181818181815,
540.0
],
[
0,
0,
1
]
]
}
To:
{
"intrinsics": [
[
1.1363636363636362,
0,
0.5
],
[
0,
2.02020202020202,
0.5
],
[
0,
0,
1
]
]
}
, having cx and cy to 0.5
Is this normalized version of the intrinsics right?
Are there any tips in finetuning such a large foundation model?
Increasing the number of iterations will also play a huge part here?
Also the weight here, if we have only a dataset plays a part?
"name": "dataset_spring",
"path": "......",
"label_type": "synthetic",
"index": ".index.txt",
"depth": "depth.png",
"weight": 10, <- what role plays this when we have only one dataset? I think it doesn't matter?
"center_augmentation": 0.25,
"fov_range_absolute": [30, 150],
"fov_range_relative": [0.5, 1.0],
"image_augmentation": ["jittering", "jpeg_loss", "blurring", "shot_noise"]
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the supplementary materials and eval section, then inspect the meta.json structure and the Hugging Face preprocessed Spring test dataset used for comparison. Reproduce the stock-versus-finetuned evaluation using the stated 1,000 iterations and recorded intrinsics, and document answers about intrinsic normalization, iteration count, and the single-dataset weight.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- huggingface
- Domain
- computer-vision, machine-learning
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100