运行 moss_cli_demo.py 出现错误,请问这是提示我cpu内存不够吗?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
D:\UserSoftware\Anaconda3\envs\Moss\python.exe E:/MOSS-main/moss_cli_demo.py
┌───────────────────── Traceback (most recent call last) ─────────────────────┐
│ E:\MOSS-main\moss_cli_demo.py:47 in │
│ │
│ 44 │ │ raw_model, model_path, device_map="auto", no_split_module_clas │
│ 45 │ ) │
│ 46 else: # on a single gpu │
│ > 47 │ model = MossForCausalLM.from_pretrained(model_path).half().cuda() │
│ 48 │
│ 49 │
│ 50 def clear(): │
│ │
│ D:\UserSoftware\Anaconda3\envs\Moss\lib\site-packages\transformers\modeling │
│ _utils.py:2629 in from_pretrained │
│ │
│ 2626 │ │ │ init_contexts.append(init_empty_weights()) │
│ 2627 │ │ │
│ 2628 │ │ with ContextManagers(init_contexts): │
│ > 2629 │ │ │ model = cls(config, *model_args, **model_kwargs) │
│ 2630 │ │ │
│ 2631 │ │ # Check first if we are from_pt │
│ 2632 │ │ if use_keep_in_fp32_modules: │
│ │
│ E:\MOSS-main\models\modeling_moss.py:607 in init │
│ │
│ 604 │ │ │ torch.set_default_dtype(torch.half) │
│ 605 │ │ │ transformers.modeling_utils._init_weights = False │
│ 606 │ │ │ torch.set_default_dtype(torch.half) │
│ > 607 │ │ self.transformer = MossModel(config) │
│ 608 │ │ self.lm_head = nn.Linear(config.n_embd, config.vocab_size) │
│ 609 │ │ if config.wbits in [4, 8]: │
│ 610 │ │ │ torch.set_default_dtype(torch.float) │
│ │
│ E:\MOSS-main\models\modeling_moss.py:401 in init │
│ │
│ 398 │ │ self.vocab_size = config.vocab_size │
│ 399 │ │ self.wte = nn.Embedding(config.vocab_size, self.embed_dim) │
│ 400 │ │ self.drop = nn.Dropout(config.embd_pdrop) │
│ > 401 │ │ self.h = nn.ModuleList([MossBlock(config) for _ in range(conf │
│ 402 │ │ self.ln_f = nn.LayerNorm(self.embed_dim, eps=config.layer_nor │
│ 403 │ │ self.rotary_dim = min(config.rotary_dim, config.n_ctx // conf │
│ 404 │
│ │
│ E:\MOSS-main\models\modeling_moss.py:401 in │
│ │
│ 398 │ │ self.vocab_size = config.vocab_size │
│ 399 │ │ self.wte = nn.Embedding(config.vocab_size, self.embed_dim) │
│ 400 │ │ self.drop = nn.Dropout(config.embd_pdrop) │
│ > 401 │ │ self.h = nn.ModuleList([MossBlock(config) for _ in range(conf │
│ 402 │ │ self.ln_f = nn.LayerNorm(self.embed_dim, eps=config.layer_nor │
│ 403 │ │ self.rotary_dim = min(config.rotary_dim, config.n_ctx // conf │
│ 404 │
│ │
│ E:\MOSS-main\models\modeling_moss.py:255 in init │
│ │
│ 252 │ │ super().init() │
│ 253 │ │ inner_dim = config.n_inner if config.n_inner is not None else │
│ 254 │ │ self.ln_1 = nn.LayerNorm(config.n_embd, eps=config.layer_norm │
│ > 255 │ │ self.attn = MossAttention(config) │
│ 256 │ │ self.mlp = MossMLP(inner_dim, config) │
│ 257 │ │
│ 258 │ def forward( │
│ │
│ E:\MOSS-main\models\modeling_moss.py:85 in init │
│ │
│ 82 │ │ │ │ f" num_attention_heads: {self.num_attention_heads}) │
│ 83 │ │ │ ) │
│ 84 │ │ self.scale_attn = torch.sqrt(torch.tensor(self.head_dim, dtyp │
│ > 85 │ │ self.qkv_proj = nn.Linear(self.embed_dim, self.embed_dim * 3, │
│ 86 │ │ │
│ 87 │ │ self.out_proj = nn.Linear(self.embed_dim, self.embed_dim, bia │
│ 88 │ │ self.rotary_dim = config.rotary_dim │
│ │
│ D:\UserSoftware\Anaconda3\envs\Moss\lib\site-packages\torch\nn\modules\line │
│ ar.py:96 in init │
│ │
│ 93 │ │ super().init() │
│ 94 │ │ self.in_features = in_features │
│ 95 │ │ self.out_features = out_features │
│ > 96 │ │ self.weight = Parameter(torch.empty((out_features, in_feature │
│ 97 │ │ if bias: │
│ 98 │ │ │ self.bias = Parameter(torch.empty(out_features, **factory │
│ 99 │ │ else: │
└─────────────────────────────────────────────────────────────────────────────┘
RuntimeError: [enforce fail at
C:\cb\pytorch_1000000000000\work\c10\core\impl\alloc_cpu.cpp:72] data.
DefaultCPUAllocator: not enough memory: you tried to allocate 226492416 bytes.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the command in moss_cli_demo.py and inspect the model-loading path at line 47. Read models/modeling_moss.py around lines 401 and 85, then verify the reported DefaultCPUAllocator memory failure in the same Anaconda and PyTorch environment. Done should be a confirmed diagnosis with a clearly documented remediation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100