LAION-AI / LAION-AI/CLIP_benchmark
additional argument not recognized in MODEL_TYPES
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 814
- Forks
- 103
- PR merge metrics
- No merged PRs in 30d
Description
Created a new file models/mrl_clip.py and updated the models/__init__.py, but the MODEL_TYPES is not updated.
from typing import Union
import torch
from .open_clip import load_open_clip
from .japanese_clip import load_japanese_clip
from .mrl_clip import load_mrl_clip
# loading function must return (model, transform, tokenizer)
TYPE2FUNC = {
"open_clip": load_open_clip,
"ja_clip": load_japanese_clip,
"mrl_clip": load_mrl_clip,
}
MODEL_TYPES = list(TYPE2FUNC.keys())
def load_clip(
model_type: str,
model_name: str,
pretrained: str,
cache_dir: str,
device: Union[str, torch.device] = "cuda"
):
assert model_type in MODEL_TYPES, f"model_type={model_type} is invalid!"
load_func = TYPE2FUNC[model_type]
return load_func(model_name=model_name, pretrained=pretrained, cache_dir=cache_dir, device=device)
Error details:
(/gscratch/krishna/arnabk1/pyclip) [arnabk1@g3041 clip_benchmark]$ python3 cli.py eval --model_type "mrl_clip" --pretrained "/gscratch/krishna/arnabk1/clip_benchmark/clip_benchmark/models/mrl_true_mp_rank_00_model_stat
es.pt" --task "zeroshot_classification" --dataset "imagenet1k"
usage: cli.py eval [-h] [--dataset DATASET [DATASET ...]] [--dataset_root DATASET_ROOT] [--split SPLIT]
[--test_split SPLIT] [--train_split TRAIN_SPLIT [TRAIN_SPLIT ...]]
[--val_split VAL_SPLIT [VAL_SPLIT ...] | --val_proportion VAL_PROPORTION
[VAL_PROPORTION ...]] [--model MODEL [MODEL ...]]
[--pretrained PRETRAINED [PRETRAINED ...]]
[--pretrained_model PRETRAINED_MODEL [PRETRAINED_MODEL ...]]
[--task {zeroshot_classification,zeroshot_retrieval,linear_probe,captioning,image_caption_selection,auto}]
[--no_amp] [--num_workers NUM_WORKERS] [--recall_k RECALL_K [RECALL_K ...]]
[--fewshot_k FEWSHOT_K] [--fewshot_epochs FEWSHOT_EPOCHS] [--fewshot_lr FEWSHOT_LR]
[--skip_load] [--distributed] [--seed SEED] [--batch_size BATCH_SIZE]
[--normalize NORMALIZE] [--model_cache_dir MODEL_CACHE_DIR]
[--feature_root FEATURE_ROOT] [--annotation_file ANNOTATION_FILE]
[--custom_classname_file CUSTOM_CLASSNAME_FILE]
[--custom_template_file CUSTOM_TEMPLATE_FILE] [--dump_classnames] [--dump_templates]
[--language LANGUAGE [LANGUAGE ...]] [--output OUTPUT] [--quiet] [--save_clf SAVE_CLF]
[--load_clfs LOAD_CLFS [LOAD_CLFS ...]] [--skip_existing]
[--model_type {open_clip,ja_clip}] [--wds_cache_dir WDS_CACHE_DIR]
cli.py eval: error: argument --model_type: invalid choice: 'mrl_clip' (choose from 'open_clip', 'ja_clip')
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in cli.py at the eval argument definition that produces the --model_type choices, then compare it with models/init.py and the MODEL_TYPES shown in the issue. The work is done when mrl_clip is accepted by the CLI and the reported evaluation command proceeds past argument parsing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- cli, machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 58/100