allenai / allenai/olmo-eval

[FEATURE] Cleanup magic flags for `olmobase:code`

未關閉
#144 0 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
enhancement
主要語言
Python
星號
70
分支
21
平均合併
2 天 11 小時
30 天內合併 PR
41

描述

## Feature Description

One issue I had running `olmobase:code` was onboarding with the proper GCS keys. (it was unclear my service credentials didn't have access, or how to get creds).

And, I had to use a few magic flags to get things running:

```
-H codex_universal \
-o provider.num_instances=4 \
-o 'sandboxes.0={"mode":"modal","instances":96,"registry_auth":{"provider":"gcp"}}' \
--secret-env ai2-tylerm_MODAL_TOKEN_ID:MODAL_TOKEN_ID \
--secret-env ai2-tylerm_MODAL_TOKEN_SECRET:MODAL_TOKEN_SECRET \
--gcp-credentials
```

It would be great if these options were fold-ed into the default behavior! (similar to our setup in oe-eval-internal)

## Proposed Solution

@undfined and I chatted briefly about this, I think he had some ideas. Seems like handling this similar to the AWS credentials might be a nice option

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。