allenai / allenai/pawls

Add CLI dataset management

未关闭
#145 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
Python
星标
435
派生
80
PR 合并指标
30 天内没有已合并 PR

描述

I was thinking of a set of commands, to create a dataset:

```bash
pawls dataset create [DATASET NAME] [INITIAL PDFS]
```

Add pdfs to the dataset:

```bash
pawls dataset add [PDFS]
```

And offer per-dataset configuration for the label-set. Some discussion happening in #144, with the proposal that datasets are top-level folders (w/in `skiff_files`). I think that's the simplest, and would let you drop an overriding configuration file into each dataset folder.

One last concern is *where* the datasets should reside. Would a user need to provide the relative path to the `skiff_files` fiolder for each of the sub-commands, to make sure they're copied into the right place?

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。