allenai / allenai/olmo-eval

[FEATURE] Cleanup magic flags for `olmobase:code`

Offen
#144 0 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
enhancement
Vorherrschende Sprache
Python
Sterne
70
Forks
21
Ø Merge
2 T. 11 Std.
Gemergte PRs (30 T.)
41

Beschreibung

## Feature Description

One issue I had running `olmobase:code` was onboarding with the proper GCS keys. (it was unclear my service credentials didn't have access, or how to get creds).

And, I had to use a few magic flags to get things running:

```
-H codex_universal \
-o provider.num_instances=4 \
-o 'sandboxes.0={"mode":"modal","instances":96,"registry_auth":{"provider":"gcp"}}' \
--secret-env ai2-tylerm_MODAL_TOKEN_ID:MODAL_TOKEN_ID \
--secret-env ai2-tylerm_MODAL_TOKEN_SECRET:MODAL_TOKEN_SECRET \
--gcp-credentials
```

It would be great if these options were fold-ed into the default behavior! (similar to our setup in oe-eval-internal)

## Proposed Solution

@undfined and I chatted briefly about this, I think he had some ideas. Seems like handling this similar to the AWS credentials might be a nice option

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.