allenai / allenai/tango

Beaker Executor opens a ton of files

Open
#486 2 comments 0 reactions 1 assignee Claimed by @epwalsh View on GitHub
bug
Dominant language
Python
Stars
572
Forks
55
PR merge metrics
No merged PRs in 30d

Description

### 🐛 Describe the bug

You have to do this in a catwalk context, on commit `e7c5d158b9e8f1c925b4894037b5371a1efdeab7`.
```Python
from tango import StepGraph
sg = StepGraph.from_file("experiments/everything/everything.jsonnet")
from tango import Workspace
ws = Workspace.from_url("beaker://ai2/task-complexity")
s = sg['trained_model_sciq_bert-base-uncased_1337']
r = ws.step_result(s)
```
At this point the Beaker Workspace will open millions of files, quickly exhausting the open file limit. Even when you Ctrl+C the proceedings, background threads keep trying to open files. Your console is now useless.

### Versions

asdasdasd

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.