microsoft / microsoft/vidur

Adding new model

Open
#32 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
682
Forks
128
Avg merge
9h 44m
Merged PRs (30d)
1

Description

The below mentioned point where the link for GPTModel is broken and can't get the exact config for the yaml file hence not able to add new model (llama2-13b) as not able to profile the data for that model.

  1. Add a YAML model config for the new model in data/model_configs.
  • Use the model's HuggingFace model id for the file name eg. data/model_configs/meta-llama/Llama-2-70b-hf.yml.
  • Refer HuggingFace config.json for the model eg. https://huggingface.co/meta-llama/Llama-2-70b-hf/blob/main/config.json.
  • Ensure that correct parameters are set in the YAML file so that the reference transformer model GPTModel closely resembles the new model.
  • We use this reference model to profile only the MLP operations of all the models so the attention operations are no-op'ed here.

Would like to know if there is any solution for this as even the model_configs folder is not present in the base directory.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by checking whether data/model_configs exists in the repository and inspect the referenced HuggingFace config.json for the llama2-13b model. Determine which YAML parameters are required for the GPTModel reference, add the model configuration, and confirm that the model can be profiled as described.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.