Support for CodeGenForCausalLM
Open
feature request
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 487
- PR merge metrics
- No merged PRs in 30d
Description
### 🚀 The feature, motivation, and pitch
I'm trying to apply RL in a code generation LM: https://huggingface.co/docs/transformers/model_doc/codegen, unfortunately getting the error below:
```
ValueError: Unsupported architecture: `CodeGenForCausalLM`. The following architectures are available for model
branching:
['GPTJForCausalLM', 'GPT2LMHeadModel', 'GPTNeoForCausalLM', 'GPTNeoXForCausalLM', 'OPTForCausalLM', 'BloomModel',
'BloomForCausalLM', 'LlamaModel', 'LlamaForCausalLM']
```
I wonder if it would be possible to add support for code modes. Thanks!
### Alternatives
_No response_
### Additional context
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.