deepseek-ai / deepseek-ai/Janus
因为需要迁移到310视频卡上，需要先转换成onnx格式,测试了多种方法都有技术阻碍
Open
- Dominant language
- Python
- Stars
- 17.8k
- Forks
- 2.2k
- PR merge metrics
- No merged PRs in 30d
Description
一种是转换成 transformers、llama、swift格式 ，然后出现了
ValueError: The checkpoint you are trying to load has model type `multi_modality` but Transformers does not recognize this architecture ，看了一下 应该是transformers 里没有配置文件中的multi_modality类别
第二种方法是onnxruntime,先启动模型，然后传输给onnxruntime,有了multi_modality类型了，
又报了
NotImplementedError: Module [MultiModalityCausalLM] is missing the required "forward" function
目前还没看懂MultiModalityCausalLM 类与AutoModelForCausalLM的forward应该如何设计，有什么好建议吗？
btw - 研化好像转换成了R1 但R1没有用MultiModalityCausalLM类
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.