python / python/typeshed

stub packages with overlapping namespaces overwrite their METADATA.toml

Open
#11,047 9 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

project: infrastructure
Dominant language
Python
Stars
5.1k
Forks
2.1k
Avg merge
1d 19h
Merged PRs (30d)
82

Description

Hello,

While working on the python3-typeshed package for Debian I noticed that if one installs two types- packages involving the same namespace, the METADATA.toml files of the first package will be overwritten by the second package:

$ python3 -m venv typeshed-venv
$ . typeshed-venv/bin/activate
$ pip install types_google_cloud_ndb
Collecting types_google_cloud_ndb
  Obtaining dependency information for types_google_cloud_ndb from https://files.pythonhosted.org/packages/09/d8/70b6b36b0e82095a43b2ff7cfe0a55f12fa53bdc70e4d2768538aae646fa/types_google_cloud_ndb-2.2.0.1-py3-none-any.whl.metadata
  Downloading types_google_cloud_ndb-2.2.0.1-py3-none-any.whl.metadata (1.6 kB)
Downloading types_google_cloud_ndb-2.2.0.1-py3-none-any.whl (16 kB)
Installing collected packages: types_google_cloud_ndb
Successfully installed types_google_cloud_ndb-2.2.0.1
$ cat typeshed-venv/lib/python3.11/site-packages/google-stubs/METADATA.toml 
version = "2.2.*"
upstream_repository = "https://github.com/googleapis/python-ndb"
partial_stub = true

[tool.stubtest]
stubtest_requirements = ["protobuf==3.20.2", "six"]
ignore_missing_stub = true
$ pip install types_protobuf
Collecting types_protobuf
  Obtaining dependency information for types_protobuf from https://files.pythonhosted.org/packages/72/03/f7dd2f1ec9712c4242f04b7cb0f7e88605a98ee2695f0e98d72a277580aa/types_protobuf-4.24.0.4-py3-none-any.whl.metadata
  Downloading types_protobuf-4.24.0.4-py3-none-any.whl.metadata (1.9 kB)
Downloading types_protobuf-4.24.0.4-py3-none-any.whl (62 kB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 62.1/62.1 kB 1.2 MB/s eta 0:00:00
Installing collected packages: types_protobuf
Successfully installed types_protobuf-4.24.0.4
$ cat typeshed-venv/lib/python3.11/site-packages/google-stubs/METADATA.toml 
version = "4.24.*"
upstream_repository = "https://github.com/protocolbuffers/protobuf"
extra_description = "Generated using [mypy-protobuf==3.5.0](https://github.com/nipunn1313/mypy-protobuf/tree/v3.5.0) on protobuf==4.21.8"
partial_stub = true

[tool.stubtest]
ignore_missing_stub = true

From a packaging system perspective, this makes me uncomfortable.

  1. Does METADATA.toml need to be installed in the top-level namespace of stub packages?
  2. Would it be a problem to exclude it from the Debian packages, instead of randomly choosing which one is kept?
  3. If keeping these files is important, perhaps they could be prefixed, so they can exist alongside each other: METADATA.google_cloud_ndb.toml / METADATA.protobuf.toml perhaps

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the two-package installation with types_google_cloud_ndb and types_protobuf, then inspect how each package installs its google-stubs/METADATA.toml file. Review the packaging behavior for stub packages and the Debian packaging concern described here. Done requires an agreed way to preserve or intentionally exclude metadata without one package overwriting another.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.