allenai / allenai/ir_datasets

warc3-wet and warc3-wet-clueweb09 are unconditional dependencies in PyPI 0.5.11 metadata

未關閉
#303 0 則留言 2 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
391
分支
58
PR 合併指標
30 天內沒有已合併 PR

描述

**Describe the bug**

The PyPI distribution of `ir-datasets==0.5.11` lists `warc3-wet` and `warc3-wet-clueweb09` as unconditional `Requires-Dist` entries, even though the [current `pyproject.toml` on `master`](https://github.com/allenai/ir_datasets/blob/master/pyproject.toml) declares them under the optional `[warc]` extra.

Inspecting the installed package metadata shows no `Provides-Extra` entries at all, and the warc packages appear as required dependencies:

```
Requires-Dist: warc3-wet>=0.2.3
Requires-Dist: warc3-wet-clueweb09>=0.2.5
```

Since these packages are licensed under GPLv2, this could potentially cause license compatibility concerns for downstream projects that depend on `ir-datasets` transitively, even if they never use WARC-related functionality.

**Affected dataset(s)**

Not specific to any dataset. This is a packaging/metadata issue affecting all installations.

**To Reproduce**

1. `pip install ir-datasets==0.5.11` (or install via any package manager)
2. Inspect the installed metadata:
```python
from importlib.metadata import metadata
m = metadata("ir-datasets")
for line in str(m).split("\n"):
if "warc" in line.lower() or "Provides-Extra" in line:
print(line)
```
3. Observe that `warc3-wet` and `warc3-wet-clueweb09` appear as unconditional `Requires-Dist`, and no `Provides-Extra` is present.

**Expected behavior**

`warc3-wet` and `warc3-wet-clueweb09` should only be installed when a user explicitly requests the `[warc]` extra (i.e., `pip install ir-datasets[warc]`), matching the intent expressed in the `pyproject.toml` on `master`.

**Additional context**

It appears that the optional dependency restructuring in `pyproject.toml` may not yet have been reflected in the 0.5.11 release build. A new release with the corrected metadata would resolve this for downstream consumers.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。