GLM-4.5-Air 部署

(ktransformers) root@63ae89fb3751:/workspace/LLM/ktransformers# python -m ktransformers.tools.export_to_gguf \
--model_path /workspace/LLM/zai-org/GLM-4.5-Air \
--output_path /workspace/LLM/GLM-4.5-Air.gguf
/workspace/miniconda3/envs/ktransformers/bin/python: Error while finding module specification for 'ktransformers.tools.export_to_gguf' (ModuleNotFoundError: No module named 'ktransformers.tools')
(ktransformers) root@63ae89fb3751:/workspace/LLM/ktransformers# python ktransformers/server/main.py --port 10110 --model_name Glm4MoeForCausalLM --model_path /workspace/LLM/zai-org/GLM-4.5-Air --optimize_config_path ktransformers/optimize/optimize_rules/Glm4Moe-serve.yaml --max_new_tokens 1024 --cache_lens 32768 --chunk_size 256 --max_batch_size 1 --backend_type balance_serve
W0814 04:18:49.584000 1889823 site-packages/torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
W0814 04:18:49.584000 1889823 site-packages/torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.
2025-08-14 04:18:49,587 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
flash_attn not found, flashinfer unit test needed it. If you are using balance serve, ignore this.
set start method
Connected to server at tcp://localhost:38125
W0814 04:18:56.954000 1890039 torch/utils/cpp_extension.py:2425] TORCH_CUDA_ARCH_LIST is not set, all archs for visible cards are included for compilation.
W0814 04:18:56.954000 1890039 torch/utils/cpp_extension.py:2425] If this is not desired, please set os.environ['TORCH_CUDA_ARCH_LIST'] to specific architectures.
2025-08-14 04:18:56,959 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
found flashinfer
flash_attn not found, flashinfer unit test needed it. If you are using balance serve, ignore this.
start method already set to spawn
Connected to server at tcp://localhost:38125
args.architectures: Glm4MoeForCausalLM
The argument `trust_remote_code` is to be used with Auto classes. It has no effect here and is ignored.
Process SpawnProcess-1:
Traceback (most recent call last):
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/process.py", line 314, in _bootstrap
self.run()
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/multiprocessing/process.py", line 108, in run
self._target(*self._args, **self._kwargs)
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 300, in run_engine
engine = Engine(args, token_queue, broadcast_endpoint, kvcache_event)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/server/backend/interfaces/balance_serve.py", line 204, in __init__
optimize_and_load_gguf(self.model, optimize_config_path, gguf_path, config)
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/optimize/optimize.py", line 126, in optimize_and_load_gguf
weights_loader = ModelLoaderFactory.create_loader(gguf_path)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/workspace/miniconda3/envs/ktransformers/lib/python3.12/site-packages/ktransformers/util/custom_loader.py", line 529, in create_loader
raise FileNotFoundError(f"Path not found: {path}")
FileNotFoundError: Path not found: /mnt/data/models/Smallthinker-21B

Reproduce the commands in the issue, then trace the failure from ktransformers/server/main.py through server/backend/interfaces/balance_serve.py, optimize/optimize.py, and util/custom_loader.py. Determine why export_to_gguf is unavailable and why the loader resolves /mnt/data/models/Smallthinker-21B instead of the supplied model; done means a documented GLM-4.5-Air deployment path that starts without these errors.

Written by the indexing model from the issue text.