bethgelab / bethgelab/sober-reasoning
Can't run code
- Dominant language
- HTML
- Stars
- 92
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
```bash
(sober) wty@ubuntu49:~/sober-reasoning$ python main.py \
> --model /data/wty/models/deepscaler_1.5b_grpo_replica_step500 \
> --task "custom|aime24|0|0" \
> --temperature 0.8 \
> --top_p 0.9 \
> --seed 0 \
> --output_dir /path/to/output \
> --max_new_tokens 4096 \
> --max_model_length 2048 \
> --custom_tasks_directory lighteval_tasks.py \
> --use_chat_template
INFO 09-08 15:04:59 __init__.py:190] Automatically detected platform cuda.
`torch_dtype` is deprecated! Use `dtype` instead!
Loading safetensors checkpoint shards: 0% Completed | 0/2 [00:00
[rank0]: main()
[rank0]: File "/home/wty/sober-reasoning/main.py", line 123, in main
[rank0]: pipeline = Pipeline(
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/lighteval/pipeline.py", line 162, in __init__
[rank0]: self._init_tasks_and_requests(tasks=tasks)
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/lighteval/pipeline.py", line 220, in _init_tasks_and_requests
[rank0]: task_names_list, fewshots_dict = taskinfo_selector(tasks, registry)
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/lighteval/tasks/registry.py", line 299, in taskinfo_selector
[rank0]: expanded_tasks = task_registry.task_groups_dict.get(maybe_task_group, [maybe_task_group])
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/lighteval/tasks/registry.py", line 190, in task_groups_dict
[rank0]: custom_tasks_module = create_custom_tasks_module(custom_tasks=self._custom_tasks)
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/lighteval/tasks/registry.py", line 258, in create_custom_tasks_module
[rank0]: dataset_module = dataset_module_factory(str(custom_tasks), trust_remote_code=True)
[rank0]: File "/home/wty/miniconda3/envs/sober/lib/python3.10/site-packages/datasets/load.py", line 878, in dataset_module_factory
[rank0]: download_config = DownloadConfig(**download_kwargs)
[rank0]: TypeError: DownloadConfig.__init__() got an unexpected keyword argument 'trust_remote_code'
[rank0]:[W908 15:05:32.355204775 ProcessGroupNCCL.cpp:1250] Warning: WARNING: process group has NOT been destroyed before we destruct ProcessGroupNCCL. On normal program exit, the application should call destroy_process_group to ensure that any pending NCCL operations have finished in this process. In rare cases this process can exit before this point and block the progress of another member of the process group. This constraint has always been present, but this warning has only been added since PyTorch 2.4 (function operator())
```
I install the same packages in the dockerfile...
but it is not working
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with main.py around the Pipeline construction at line 123, then compare the Dockerfile dependency versions with the lighteval and datasets calls shown in the traceback. Re-run the provided command and verify that custom task initialization proceeds without the DownloadConfig TypeError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- dockerfile, python, pytorch
- Domain
- devops, machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100