Add support for zai-org/GLM-4.1V-9B-Thinking
Open
Nobody has claimed this yet.
model support request
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 315
- Avg merge
- 4d 11h
- Merged PRs (30d)
- 16
Description
Add support for zai-org/GLM-4.1V-9B-Thinking. When I run:
olive optimize --model_name_or_path zai-org/GLM-4.1V-9B-Thinking --precision int4 --output_path models/glm4.1v-thinking
Got:
Loading HuggingFace model from zai-org/GLM-4.1V-9B-Thinking
Traceback (most recent call last):
File "/usr/local/bin/olive", line 8, in <module>
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/cli/launcher.py", line 68, in main
service.run()
File "/usr/local/lib/python3.11/site-packages/olive/cli/optimize.py", line 197, in run
return self._run_workflow()
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/cli/base.py", line 44, in _run_workflow
workflow_output = olive_run(run_config)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/workflows/run/run.py", line 261, in run
return run_engine(package_config, run_config)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/workflows/run/run.py", line 200, in run_engine
return engine.run(
^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/engine/engine.py", line 231, in run
run_result = self.run_accelerator(
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/engine/engine.py", line 314, in run_accelerator
output_footprint = self._run_no_search(input_model_config, input_model_id, accelerator_spec, output_dir)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/engine/engine.py", line 358, in _run_no_search
should_prune, signal, model_ids = self._run_passes(input_model_config, input_model_id, accelerator_spec)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/engine/engine.py", line 642, in _run_passes
model_config, model_id = self._run_pass(
^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/engine/engine.py", line 743, in _run_pass
output_model_config = host.run_pass(p, input_model_config, output_model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/systems/local.py", line 45, in run_pass
output_model = the_pass.run(model, output_model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/passes/olive_pass.py", line 242, in run
output_model = self._run_for_config(model, self.config, output_model_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/passes/pytorch/gptq.py", line 121, in _run_for_config
wrapper = ModelWrapper.from_model(load_hf_base_model(model, torch_dtype="auto"))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/passes/pytorch/train_utils.py", line 126, in load_hf_base_model
return new_model_handler.load_model(cache_model=False)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/model/handler/hf.py", line 75, in load_model
model = load_model_from_task(self.task, self.model_path, **self.get_load_kwargs())
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/common/hf/utils.py", line 62, in load_model_from_task
model = from_pretrained(model_class, model_name_or_path, "model", **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/olive/common/hf/utils.py", line 94, in from_pretrained
return cls.from_pretrained(get_pretrained_name_or_path(model_name_or_path, mlflow_dir), **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/auto/auto_factory.py", line 603, in from_pretrained
raise ValueError(
ValueError: Unrecognized configuration class <class 'transformers.models.glm4v.configuration_glm4v.Glm4vConfig'> for this kind of AutoModel: AutoModelForCausalLM.
Model type should be one of ArceeConfig, AriaTextConfig, BambaConfig, BartConfig, BertConfig, BertGenerationConfig, BigBirdConfig, BigBirdPegasusConfig, BioGptConfig, BitNetConfig, BlenderbotConfig, BlenderbotSmallConfig, BloomConfig, CamembertConfig, LlamaConfig, CodeGenConfig, CohereConfig, Cohere2Config, CpmAntConfig, CTRLConfig, Data2VecTextConfig, DbrxConfig, DeepseekV2Config, DeepseekV3Config, DiffLlamaConfig, DogeConfig, Dots1Config, ElectraConfig, Emu3Config, ErnieConfig, Ernie4_5Config, Ernie4_5_MoeConfig, Exaone4Config, FalconConfig, FalconH1Config, FalconMambaConfig, FuyuConfig, GemmaConfig, Gemma2Config, Gemma3Config, Gemma3TextConfig, Gemma3nConfig, Gemma3nTextConfig, GitConfig, GlmConfig, Glm4Config, Glm4MoeConfig, GotOcr2Config, GPT2Config, GPT2Config, GPTBigCodeConfig, GPTNeoConfig, GPTNeoXConfig, GPTNeoXJapaneseConfig, GptOssConfig, GPTJConfig, GraniteConfig, GraniteMoeConfig, GraniteMoeHybridConfig, GraniteMoeSharedConfig, HeliumConfig, JambaConfig, JetMoeConfig, Lfm2Config, LlamaConfig, Llama4Config, Llama4TextConfig, MambaConfig, Mamba2Config, MarianConfig, MBartConfig, MegaConfig, MegatronBertConfig, MiniMaxConfig, MistralConfig, MixtralConfig, MllamaConfig, ModernBertDecoderConfig, MoshiConfig, MptConfig, MusicgenConfig, MusicgenMelodyConfig, MvpConfig, NemotronConfig, OlmoConfig, Olmo2Config, OlmoeConfig, OpenLlamaConfig, OpenAIGPTConfig, OPTConfig, PegasusConfig, PersimmonConfig, PhiConfig, Phi3Config, Phi4MultimodalConfig, PhimoeConfig, PLBartConfig, ProphetNetConfig, QDQBertConfig, Qwen2Config, Qwen2MoeConfig, Qwen3Config, Qwen3MoeConfig, RecurrentGemmaConfig, ReformerConfig, RemBertConfig, RobertaConfig, RobertaPreLayerNormConfig, RoCBertConfig, RoFormerConfig, RwkvConfig, SmolLM3Config, Speech2Text2Config, StableLmConfig, Starcoder2Config, TransfoXLConfig, TrOCRConfig, WhisperConfig, XGLMConfig, XLMConfig, XLMProphetNetConfig, XLMRobertaConfig, XLMRobertaXLConfig, XLNetConfig, xLSTMConfig, XmodConfig, ZambaConfig, Zamba2Config.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the command and start by tracing model loading through olive/model/handler/hf.py and olive/common/hf/utils.py, then inspect the GPTQ path in olive/passes/pytorch/gptq.py. Determine how the reported Glm4vConfig is handled by the existing loader. Done means the supplied optimization command loads zai-org/GLM-4.1V-9B-Thinking without the AutoModelForCausalLM configuration error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100