allenai / allenai/olmes

Argument list too long

Ouverte
#26 0 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
395
Forks
105
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

Hi,

Thanks for the great repo, while attempting to reproduce instruct results, using olmo:adapt for tasks, it runs into an error due to length of the config exceding the available bash arg length which is around 6 MB for the machine I am working on. The solution is trivial .

Adding following codeblock before line 379 in launch.py
```
import json, tempfile
tf = tempfile.NamedTemporaryFile(mode='w', suffix='.json', delete=False)
json.dump(task_configs, tf)
tf.close()
run_eval_args["task"] = [tf.name]
```

Replacing below codeblock with the for loop in line 332 in launch.py
```
for task in tasks:
if task.endswith('.json') and os.path.exists(task):
import json
task_configs.extend(json.load(open(task)))
os.remove(task) if '/tmp/' in task else None
else:
task_config = parse_args_string(task, "task_name")
task_configs.append(get_dict_with_defaults(task_config, task_config_shared))
```

Above changes resolves the issue however I havent fully tested yet across possible different configs, will create a PR if it is stable.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.