ashutosh1919 / ashutosh1919/data2vec-pytorch

PermissionError!!! PermissionError: [Errno 13] Permission denied: 'data/train.raw'

Open
#4 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
16
Forks
3
PR merge metrics
No merged PRs in 30d

Description

Although I have given file permissions, an error still occurs when running the script. The error location has been marked in bold. Can you help me solve it?please

(pytorch1.10.0) E:\Code\notebooks>bash scripts/train_data2vec_multi_text.sh
Traceback (most recent call last):
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 150, in
main()
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 69, in main
inputs = [
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 70, in
stack.enter_context(open(input, "r", encoding="utf-8"))
**PermissionError: [Errno 13] Permission denied: 'data/train.raw'**

Traceback (most recent call last):
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 150, in
main()
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 69, in main
inputs = [
File "E:\Code\notebooks\datasets\openwebtext\multiprocessing_bpe_encoder.py", line 70, in
stack.enter_context(open(input, "r", encoding="utf-8"))
**PermissionError: [Errno 13] Permission denied: 'data/valid.raw'**
2024-03-11 11:01:49 | INFO | fairseq_cli.preprocess | Namespace(aim_repo=None, aim_run_hash=None, align_suffix=None, alignfile=None, all_gather_list_size=16384, amp=Fal
se, amp_batch_retries=2, amp_init_scale=128, amp_scale_window=None, azureml_logging=False, bf16=False, bpe=None, cpu=False, criterion='cross_entropy', dataset_impl='mma
p', destdir='final_data/', dict_only=False, empty_cache_freq=0, fp16=False, fp16_init_scale=128, fp16_no_flatten_grads=False, fp16_scale_tolerance=0.0, fp16_scale_windo
w=None, joined_dictionary=False, log_file=None, log_format=None, log_interval=100, lr_scheduler='fixed', memory_efficient_bf16=False, memory_efficient_fp16=False, min_l
oss_scale=0.0001, model_parallel_size=1, no_progress_bar=False, nwordssrc=-1, nwordstgt=-1, on_cpu_convert_precision=False, only_source=True, optimizer=None, padding_fa
ctor=8, plasma_path='/tmp/plasma', profile=False, quantization_config_path=None, reset_logging=False, scoring='bleu', seed=1, source_lang=None, srcdict=None, suppress_c
rashes=False, target_lang=None, task='translation', tensorboard_logdir=None, testpref=None, tgtdict=None, threshold_loss_scale=None, thresholdsrc=0, thresholdtgt=0, tok
enizer=None, tpu=False, trainpref='data/train.bpe', use_plasma_view=False, user_dir=None, validpref='data/valid.bpe', wandb_project=None, workers=100)
Traceback (most recent call last):
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "E:\anaconda3\envs\pytorch1.10.0\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
File "E:\anaconda3\envs\pytorch1.10.0\Scripts\fairseq-preprocess.exe\__main__.py", line 7, in
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq_cli\preprocess.py", line 389, in cli_main
main(args)
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq_cli\preprocess.py", line 340, in main
src_dict = _build_dictionary(
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq_cli\preprocess.py", line 87, in _build_dictionary
return task.build_dictionary(
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq\tasks\fairseq_task.py", line 121, in build_dictionary
Dictionary.add_file_to_dictionary(
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq\data\dictionary.py", line 354, in add_file_to_dictionary
offsets = find_offsets(local_file, num_workers)
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq\file_chunker_utils.py", line 25, in find_offsets
with open(filename, "r", encoding="utf-8") as f:
**PermissionError: [Errno 13] Permission denied: 'data/train.bpe'**
Traceback (most recent call last):
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq_cli\hydra_train.py", line 27, in hydra_main
_hydra_main(cfg)
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq_cli\hydra_train.py", line 31, in _hydra_main
add_defaults(cfg)
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq\dataclass\initialize.py", line 61, in add_defaults
cfg[k] = merge_with_parent(dc, field_cfg)
File "E:\anaconda3\envs\pytorch1.10.0\lib\site-packages\fairseq\dataclass\utils.py", line 500, in merge_with_parent
merged_cfg = OmegaConf.merge(dc, cfg)
omegaconf.errors.ConfigKeyError: Key 'include_index' not in 'MaskedLMConfig'
full_key: include_index
reference_type=Optional[MaskedLMConfig]
object_type=MaskedLMConfig

Set the environment variable HYDRA_FULL_ERROR=1 for a complete stack trace.
scripts/train_data2vec_multi_text.sh: line 11: distributed_training.distributed_world_size=1: command not found

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing bash scripts/train_data2vec_multi_text.sh in the reported PyTorch 1.10.0 environment, then inspect datasets/openwebtext/multiprocessing_bpe_encoder.py around lines 69-70 and the script's line 11. Compare the reported PermissionError, Hydra ConfigKeyError, and shell error with the script inputs and installed versions; done means the command runs without these failures or the required environment and incompatibilities are documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
bash, python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.