`UnpicklingError: Weights only load failed. This file can still be loaded` occured on document's code of [tacotron2_pipeline_tutorial.html]
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.9k
- Forks
- 799
- Avg merge
- 58m
- Merged PRs (30d)
- 3
Description
🐛 Describe the bug
This code make error.
It's occured on dp package. so I don't know exactly which should be made a issue. But anyway this documentaion fails.
UnpicklingError Traceback (most recent call last)
Cell In[6], line 7
3 bundle = torchaudio.pipelines.TACOTRON2_WAVERNN_PHONE_LJSPEECH
4 # torch.serialization.safe_globals([bundle.get_text_processor.name])
5 # torch.serialization.add_safe_globals([Preprocessor])
6 # with torch.serialization.safe_globals([Preprocessor]): # Allowlist the Preprocessor class
----> 7 processor = bundle.get_text_processor()
8 text = "Hello world! Text to speech!"
9 with torch.inference_mode():
File /opt/conda/lib/python3.10/site-packages/torchaudio/pipelines/_tts/impl.py:140, in _PhoneMixin.get_text_processor(self, dl_kwargs)
139 def get_text_processor(self, *, dl_kwargs=None) -> Tacotron2TTSBundle.TextProcessor:
--> 140 return _EnglishPhoneProcessor(dl_kwargs=dl_kwargs)
File /opt/conda/lib/python3.10/site-packages/torchaudio/pipelines/_tts/impl.py:47, in _EnglishPhoneProcessor.init(self, dl_kwargs)
45 self._tokens = utils._get_phones()
46 self._mapping = {p: i for i, p in enumerate(self._tokens)}
---> 47 self._phonemizer = utils._load_phonemizer("en_us_cmudict_forward.pt", dl_kwargs=dl_kwargs)
48 self.pattern = r"([[A-Z]+?]|[!'(),.:;? -])"
File /opt/conda/lib/python3.10/site-packages/torchaudio/pipelines/_tts/utils.py:177, in _load_phonemizer(file, dl_kwargs)
175 dl_kwargs = {} if dl_kwargs is None else dl_kwargs
176 download_url_to_file(url, path, **dl_kwargs)
--> 177 return Phonemizer.from_checkpoint(path)
178 finally:
179 logger.setLevel(orig_level)
File /opt/conda/lib/python3.10/site-packages/dp/phonemizer.py:202, in Phonemizer.from_checkpoint(cls, checkpoint_path, device, lang_phoneme_dict)
186 @classmethod
187 def from_checkpoint(cls,
188 checkpoint_path: str,
189 device='cpu',
190 lang_phoneme_dict: Dict[str, Dict[str, str]] = None) -> 'Phonemizer':
191 """Initializes a Phonemizer object from a model checkpoint (.pt file).
192
193 Args:
(...)
199 Phonemizer: Phonemizer object carrying the loaded model and, optionally, a phoneme dictionary.
200 """
--> 202 model, checkpoint = load_checkpoint(checkpoint_path, device=device)
203 applied_phoneme_dict = None
204 if lang_phoneme_dict is not None:
File /opt/conda/lib/python3.10/site-packages/dp/model/model.py:306, in load_checkpoint(checkpoint_path, device)
294 """
295 Initializes a model from a checkpoint (.pt file).
296
(...)
302 and the second element is a dictionary (config).
303 """
305 device = torch.device(device)
--> 306 checkpoint = torch.load(checkpoint_path, map_location=device)
307 model_type = checkpoint['config']['model']['type']
308 model_type = ModelType(model_type)
File /opt/conda/lib/python3.10/site-packages/torch/serialization.py:1524, in load(f, map_location, pickle_module, weights_only, mmap, **pickle_load_args)
1516 return _load(
1517 opened_zipfile,
1518 map_location,
(...)
1521 **pickle_load_args,
1522 )
1523 except pickle.UnpicklingError as e:
-> 1524 raise pickle.UnpicklingError(_get_wo_message(str(e))) from None
1525 return _load(
1526 opened_zipfile,
1527 map_location,
(...)
1530 **pickle_load_args,
1531 )
1532 if mmap:
UnpicklingError: Weights only load failed. This file can still be loaded, to do so you have two options, do those steps only if you trust the source of the checkpoint.
(1) In PyTorch 2.6, we changed the default value of the weights_only argument in torch.load from False to True. Re-running torch.load with weights_only set to False will likely succeed, but it can result in arbitrary code execution. Do it only if you got the file from a trusted source.
(2) Alternatively, to load with weights_only=True please check the recommended steps in the following error message.
WeightsUnpickler error: Unsupported global: GLOBAL dp.preprocessing.text.Preprocessor was not an allowed global by default. Please use torch.serialization.add_safe_globals([dp.preprocessing.text.Preprocessor]) or the torch.serialization.safe_globals([dp.preprocessing.text.Preprocessor]) context manager to allowlist this global if you trust this class/function.
Check the documentation of torch.load to learn more about types accepted by default with weights_only https://pytorch.org/docs/stable/generated/torch.load.html.
Versions
Collecting environment information...
PyTorch version: 2.7.0+cu126
Is debug build: False
CUDA used to build PyTorch: 12.6
ROCM used to build PyTorch: N/A
OS: Ubuntu 20.04.6 LTS (x86_64)
GCC version: (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0
Clang version: Could not collect
CMake version: version 3.26.4
Libc version: glibc-2.31
Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0] (64-bit runtime)
Python platform: Linux-5.4.0-166-generic-x86_64-with-glibc2.31
Is CUDA available: True
CUDA runtime version: Could not collect
CUDA_MODULE_LOADING set to: LAZY
GPU models and configuration: GPU 0: NVIDIA GeForce RTX 3090
Nvidia driver version: 535.86.10
cuDNN version: Could not collect
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True
CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
Address sizes: 43 bits physical, 48 bits virtual
CPU(s): 48
On-line CPU(s) list: 0-47
Thread(s) per core: 2
Core(s) per socket: 24
Socket(s): 1
NUMA node(s): 1
Vendor ID: AuthenticAMD
CPU family: 23
Model: 49
Model name: AMD Ryzen Threadripper 3960X 24-Core Processor
Stepping: 0
Frequency boost: enabled
CPU MHz: 2215.582
CPU max MHz: 3800.0000
CPU min MHz: 2200.0000
BogoMIPS: 7600.21
Virtualization: AMD-V
L1d cache: 768 KiB
L1i cache: 768 KiB
L2 cache: 12 MiB
L3 cache: 128 MiB
NUMA node0 CPU(s): 0-47
Vulnerability Gather data sampling: Not affected
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Mmio stale data: Not affected
Vulnerability Retbleed: Vulnerable
Vulnerability Spec store bypass: Mitigation; Speculative Store Bypass disabled via prctl and seccomp
Vulnerability Spectre v1: Mitigation; usercopy/swapgs barriers and __user pointer sanitization
Vulnerability Spectre v2: Mitigation; Retpolines, IBPB conditional, STIBP conditional, RSB filling, PBRSB-eIBRS Not affected
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid aperfmperf pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx cpb cat_l3 cdp_l3 hw_pstate ssbd mba ibpb stibp vmmcall fsgsbase bmi1 avx2 smep bmi2 cqm rdt_a rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves cqm_llc cqm_occup_llc cqm_mbm_total cqm_mbm_local clzero irperf xsaveerptr wbnoinvd arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif umip rdpid overflow_recov succor smca sme sev sev_es
Versions of relevant libraries:
[pip3] numpy==1.23.5
[pip3] nvidia-cublas-cu12==12.6.4.1
[pip3] nvidia-cuda-cupti-cu12==12.6.80
[pip3] nvidia-cuda-nvrtc-cu12==12.6.77
[pip3] nvidia-cuda-runtime-cu12==12.6.77
[pip3] nvidia-cudnn-cu12==9.5.1.17
[pip3] nvidia-cufft-cu12==11.3.0.4
[pip3] nvidia-curand-cu12==10.3.7.77
[pip3] nvidia-cusolver-cu12==11.7.1.2
[pip3] nvidia-cusparse-cu12==12.5.4.2
[pip3] nvidia-cusparselt-cu12==0.6.3
[pip3] nvidia-nccl-cu12==2.26.2
[pip3] nvidia-nvjitlink-cu12==12.6.85
[pip3] nvidia-nvtx-cu12==12.6.77
[pip3] optree==0.15.0
[pip3] torch==2.7.0
[pip3] torch-complex==0.4.4
[pip3] torchaudio==2.7.0
[pip3] torchelastic==0.2.2
[pip3] torchvision==0.16.0
[pip3] triton==3.3.0
[conda] blas 1.0 mkl
[conda] cuda-cudart 11.8.89 0 nvidia
[conda] cuda-cupti 11.8.87 0 nvidia
[conda] cuda-libraries 11.8.0 0 nvidia
[conda] cuda-nvrtc 11.8.89 0 nvidia
[conda] cuda-nvtx 11.8.86 0 nvidia
[conda] cuda-runtime 11.8.0 0 nvidia
[conda] ffmpeg 4.3 hf484d3e_0 pytorch
[conda] libcublas 11.11.3.6 0 nvidia
[conda] libcufft 10.9.0.58 0 nvidia
[conda] libcurand 10.3.3.141 0 nvidia
[conda] libcusolver 11.4.1.48 0 nvidia
[conda] libcusparse 11.7.5.86 0 nvidia
[conda] libjpeg-turbo 2.0.0 h9bf148f_0 pytorch
[conda] mkl 2023.1.0 h213fc3f_46343
[conda] mkl-service 2.4.0 py310h5eee18b_1
[conda] mkl_fft 1.3.8 py310h5eee18b_0
[conda] mkl_random 1.2.4 py310hdb19cb5_0
[conda] numpy 1.23.5 pypi_0 pypi
[conda] nvidia-cublas-cu12 12.6.4.1 pypi_0 pypi
[conda] nvidia-cuda-cupti-cu12 12.6.80 pypi_0 pypi
[conda] nvidia-cuda-nvrtc-cu12 12.6.77 pypi_0 pypi
[conda] nvidia-cuda-runtime-cu12 12.6.77 pypi_0 pypi
[conda] nvidia-cudnn-cu12 9.5.1.17 pypi_0 pypi
[conda] nvidia-cufft-cu12 11.3.0.4 pypi_0 pypi
[conda] nvidia-curand-cu12 10.3.7.77 pypi_0 pypi
[conda] nvidia-cusolver-cu12 11.7.1.2 pypi_0 pypi
[conda] nvidia-cusparse-cu12 12.5.4.2 pypi_0 pypi
[conda] nvidia-cusparselt-cu12 0.6.3 pypi_0 pypi
[conda] nvidia-nccl-cu12 2.26.2 pypi_0 pypi
[conda] nvidia-nvjitlink-cu12 12.6.85 pypi_0 pypi
[conda] nvidia-nvtx-cu12 12.6.77 pypi_0 pypi
[conda] optree 0.15.0 pypi_0 pypi
[conda] pytorch-cuda 11.8 h7e8668a_5 pytorch
[conda] pytorch-mutex 1.0 cuda pytorch
[conda] torch 2.7.0 pypi_0 pypi
[conda] torch-complex 0.4.4 pypi_0 pypi
[conda] torchaudio 2.7.0 pypi_0 pypi
[conda] torchelastic 0.2.2 pypi_0 pypi
[conda] torchvision 0.16.0 py310_cu118 pytorch
[conda] triton 3.3.0 pypi_0 pypi
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with examples/tutorials/tacotron2_pipeline_tutorial.py at lines 164-173 and reproduce the failure at bundle.get_text_processor(). Trace the call into torchaudio's TTS implementation and the dp checkpoint loader shown in the report. Done means the tutorial's text-processor step runs successfully with the supported dependency setup, with the fix or compatibility requirement documented and tested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- documentation, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100