Dataset loading issue for german_rag_evals on Windows

Hello, I don't know what I'm doing wrong. I received the following error as indicated in the title.

My input was as shown on this website: :
[Hugging Face - Ger-RAG-eval](https://huggingface.co/datasets/deutsche-telekom/Ger-RAG-eval).

python run_evals_accelerate.py ^
--model_args "pretrained=DiscoResearch/DiscoLM_German_7b_v1" ^
--tasks "./examples/tasks/all_german_rag_evals.txt" ^
--override_batch_size 1 ^
--use_chat_template ^
--custom_tasks "community_tasks/german_rag_evals.py" ^
--output_dir "./evals/"
The output was as follows:

INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
INFO:absl:Using default tokenizer.
WARNING:bitsandbytes.cextension:The installed version of bitsandbytes was compiled without GPU support. 8-bit optimizers, 8-bit multiplication, and GPU quantization are unavailable.
WARNING:lighteval.logging.hierarchical_logger:main: (0, Namespace(model_config_path=None, model_args='pretrained=DiscoResearch/DiscoLM_German_7b_v1', max_samples=None, override_batch_size=1, job_id='', output_dir='./evals/', push_results_to_hub=False, save_details=False, push_details_to_hub=False, public_run=False, cache_dir=None, results_org=None, use_chat_template=True, system_prompt=None, dataset_loading_processes=1, custom_tasks='community_tasks/german_rag_evals.py', tasks='./examples/tasks/all_german_rag_evals.txt', num_fewshot_seeds=1)), {
WARNING:lighteval.logging.hierarchical_logger: Test all gather {
WARNING:lighteval.logging.hierarchical_logger: Test gather tensor
WARNING:lighteval.logging.hierarchical_logger: gathered_tensor tensor([0]), should be [0]
WARNING:lighteval.logging.hierarchical_logger: } [0:00:00.010932]
WARNING:lighteval.logging.hierarchical_logger: Creating model configuration {
WARNING:lighteval.logging.hierarchical_logger: } [0:00:00]
WARNING:lighteval.logging.hierarchical_logger: Model loading {
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
WARNING:lighteval.logging.hierarchical_logger: Tokenizer truncation and padding size set to the left side.
WARNING:lighteval.logging.hierarchical_logger: We are not in a distributed setting. Setting model_parallel to False.
WARNING:lighteval.logging.hierarchical_logger: Model parallel was set to False, max memory set to None and device map to None
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:31<00:00, 10.60s/it]
WARNING:lighteval.logging.hierarchical_logger: Using Data Parallelism, putting model on device cpu
WARNING:lighteval.logging.hierarchical_logger: Model info: ModelInfo(model_name='DiscoResearch/DiscoLM_German_7b_v1', model_sha='560f972f9f735fc9289584b3aa8d75d0e539c44e', model_dtype='torch.bfloat16', model_size='13.49 GB')
WARNING:lighteval.logging.hierarchical_logger: } [0:00:33.371562]
WARNING:lighteval.logging.hierarchical_logger: Tasks loading {
WARNING:lighteval.logging.hierarchical_logger: } [0:00:01.405496]
WARNING:lighteval.logging.hierarchical_logger:} [0:00:34.806011]
Traceback (most recent call last):
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\datasets\load.py", line 117, in resolve_trust_remote_code
signal.signal(signal.SIGALRM, _raise_timeout_error)
AttributeError: module 'signal' has no attribute 'SIGALRM'. Did you mean: 'SIGABRT'?

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "D:\Arbeit\AIUI\RAG Telecom Dataset\lighteval\run_evals_accelerate.py", line 82, in
main(args)
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\lighteval\logging\hierarchical_logger.py", line 166, in wrapper
return fn(*args, **kwargs)
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\lighteval\main_accelerate.py", line 83, in main
task_dict = Registry(cache_dir=env_config.cache_dir).get_task_dict(
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\lighteval\tasks\registry.py", line 135, in get_task_dict
custom_tasks_module.append(create_custom_tasks_module(custom_tasks=custom_tasks))
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\lighteval\tasks\registry.py", line 170, in create_custom_tasks_module
dataset_module = dataset_module_factory(str(custom_tasks))
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\datasets\load.py", line 1814, in dataset_module_factory
).get_module()
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\datasets\load.py", line 962, in get_module
trust_remote_code = resolve_trust_remote_code(self.trust_remote_code, self.name)
File "C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\lib\site-packages\datasets\load.py", line 133, in resolve_trust_remote_code
raise ValueError(
ValueError: The repository for german_rag_evals contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/german_rag_evals.
Please pass the argument `trust_remote_code=True` to allow custom code to be run.

I discovered that the argument `trust_remote_code=True` must be passed as part of the model_args parameter. To fix the issue, I tried the following code, but unfortunately, the error persisted.

python run_evals_accelerate.py ^
--model_args "pretrained=DiscoResearch/DiscoLM_German_7b_v1,trust_remote_code=True" ^
--tasks "./examples/tasks/all_german_rag_evals.txt" ^
--override_batch_size 1 ^
--use_chat_template ^
--custom_tasks "community_tasks/german_rag_evals.py" ^
--output_dir "./evals/"

Maybe this can help.

When I entered the command `accelerate env`, I received the following output:

Copy-and-paste the text below in your GitHub issue

- Accelerate version: 0.31.0
- Platform: Windows-10-10.0.19045-SP0
- accelerate bash location: C:\apps\entwicklungsumgebung\anaconda3\envs\lighteval\Scripts\accelerate.exe
- Python version: 3.10.14
- Numpy version: 1.26.4
- PyTorch version (GPU?): 2.3.1+cpu (False)
- PyTorch XPU available: False
- PyTorch NPU available: False
- PyTorch MLU available: False
- System RAM: 15.90 GB
- Accelerate default config:
- compute_environment: LOCAL_MACHINE
- distributed_type: NO
- mixed_precision: no
- use_cpu: False
- debug: False
- num_processes: 1
- machine_rank: 0
- num_machines: 1
- gpu_ids: 0
- rdzv_backend: static
- same_network: True
- main_training_function: main
- enable_cpu_affinity: False
- downcast_bf16: no
- tpu_use_cluster: False
- tpu_use_sudo: False
- tpu_env: []