allenai / allenai/ir_datasets

warc3-wet and warc3-wet-clueweb09 are unconditional dependencies in PyPI 0.5.11 metadata

Ouverte
#303 0 commentaires 2 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Python
Étoiles
391
Forks
58
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

**Describe the bug**

The PyPI distribution of `ir-datasets==0.5.11` lists `warc3-wet` and `warc3-wet-clueweb09` as unconditional `Requires-Dist` entries, even though the [current `pyproject.toml` on `master`](https://github.com/allenai/ir_datasets/blob/master/pyproject.toml) declares them under the optional `[warc]` extra.

Inspecting the installed package metadata shows no `Provides-Extra` entries at all, and the warc packages appear as required dependencies:

```
Requires-Dist: warc3-wet>=0.2.3
Requires-Dist: warc3-wet-clueweb09>=0.2.5
```

Since these packages are licensed under GPLv2, this could potentially cause license compatibility concerns for downstream projects that depend on `ir-datasets` transitively, even if they never use WARC-related functionality.

**Affected dataset(s)**

Not specific to any dataset. This is a packaging/metadata issue affecting all installations.

**To Reproduce**

1. `pip install ir-datasets==0.5.11` (or install via any package manager)
2. Inspect the installed metadata:
```python
from importlib.metadata import metadata
m = metadata("ir-datasets")
for line in str(m).split("\n"):
if "warc" in line.lower() or "Provides-Extra" in line:
print(line)
```
3. Observe that `warc3-wet` and `warc3-wet-clueweb09` appear as unconditional `Requires-Dist`, and no `Provides-Extra` is present.

**Expected behavior**

`warc3-wet` and `warc3-wet-clueweb09` should only be installed when a user explicitly requests the `[warc]` extra (i.e., `pip install ir-datasets[warc]`), matching the intent expressed in the `pyproject.toml` on `master`.

**Additional context**

It appears that the optional dependency restructuring in `pyproject.toml` may not yet have been reflected in the 0.5.11 release build. A new release with the corrected metadata would resolve this for downstream consumers.

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.