LAION-AI / LAION-AI/CLIP_benchmark

additional argument not recognized in MODEL_TYPES

Open
#123 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
814
Forks
103
PR merge metrics
No merged PRs in 30d

Description

Created a new file models/mrl_clip.py and updated the models/__init__.py, but the MODEL_TYPES is not updated.

from typing import Union
import torch
from .open_clip import load_open_clip
from .japanese_clip import load_japanese_clip
from .mrl_clip import load_mrl_clip

# loading function must return (model, transform, tokenizer)
TYPE2FUNC = {
    "open_clip": load_open_clip,
    "ja_clip": load_japanese_clip,
    "mrl_clip": load_mrl_clip,
}
MODEL_TYPES = list(TYPE2FUNC.keys())


def load_clip(
        model_type: str,
        model_name: str,
        pretrained: str,
        cache_dir: str,
        device: Union[str, torch.device] = "cuda"
):
    assert model_type in MODEL_TYPES, f"model_type={model_type} is invalid!"
    load_func = TYPE2FUNC[model_type]
    return load_func(model_name=model_name, pretrained=pretrained, cache_dir=cache_dir, device=device)

Error details:

(/gscratch/krishna/arnabk1/pyclip) [arnabk1@g3041 clip_benchmark]$ python3 cli.py eval --model_type "mrl_clip" --pretrained "/gscratch/krishna/arnabk1/clip_benchmark/clip_benchmark/models/mrl_true_mp_rank_00_model_stat
es.pt" --task "zeroshot_classification" --dataset "imagenet1k"
usage: cli.py eval [-h] [--dataset DATASET [DATASET ...]] [--dataset_root DATASET_ROOT] [--split SPLIT]
                   [--test_split SPLIT] [--train_split TRAIN_SPLIT [TRAIN_SPLIT ...]]
                   [--val_split VAL_SPLIT [VAL_SPLIT ...] | --val_proportion VAL_PROPORTION
                   [VAL_PROPORTION ...]] [--model MODEL [MODEL ...]]
                   [--pretrained PRETRAINED [PRETRAINED ...]]
                   [--pretrained_model PRETRAINED_MODEL [PRETRAINED_MODEL ...]]
                   [--task {zeroshot_classification,zeroshot_retrieval,linear_probe,captioning,image_caption_selection,auto}]
                   [--no_amp] [--num_workers NUM_WORKERS] [--recall_k RECALL_K [RECALL_K ...]]
                   [--fewshot_k FEWSHOT_K] [--fewshot_epochs FEWSHOT_EPOCHS] [--fewshot_lr FEWSHOT_LR]
                   [--skip_load] [--distributed] [--seed SEED] [--batch_size BATCH_SIZE]
                   [--normalize NORMALIZE] [--model_cache_dir MODEL_CACHE_DIR]
                   [--feature_root FEATURE_ROOT] [--annotation_file ANNOTATION_FILE]
                   [--custom_classname_file CUSTOM_CLASSNAME_FILE]
                   [--custom_template_file CUSTOM_TEMPLATE_FILE] [--dump_classnames] [--dump_templates]
                   [--language LANGUAGE [LANGUAGE ...]] [--output OUTPUT] [--quiet] [--save_clf SAVE_CLF]
                   [--load_clfs LOAD_CLFS [LOAD_CLFS ...]] [--skip_existing]
                   [--model_type {open_clip,ja_clip}] [--wds_cache_dir WDS_CACHE_DIR]
cli.py eval: error: argument --model_type: invalid choice: 'mrl_clip' (choose from 'open_clip', 'ja_clip')

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in cli.py at the eval argument definition that produces the --model_type choices, then compare it with models/init.py and the MODEL_TYPES shown in the issue. The work is done when mrl_clip is accepted by the CLI and the reported evaluation command proceeds past argument parsing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
cli, machine-learning
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.