python / python/typeshed

Preventing code quality regressions

Open
#11,769 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

project: infrastructure
Dominant language
Python
Stars
5.1k
Forks
2.1k
Avg merge
1d 19h
Merged PRs (30d)
82

Description

For some time now we've used pyright's stricter configuration to prevent code quality regressions. Unless a stubs package is mentioned in the config file, it's expected to have annotations for all its fields. While this has served us well for a while, it has become unsuitable since the introduction of _typeshed.Incomplete:

  • pyright doesn't understand that a (partial) Incomplete annotation is equivalent (for our purposes) to an unannotated item.
  • It's tedious to maintain the pyright stricter configuration in addition to the metadata in the stubs directory.
  • It's easy to cheat the regression check by using Incomplete.

To improve the situation, I propose to move to a custom solution:

  1. Add an incomplete marker to METADATA.toml (defaulting to false).
  2. Add a custom script to CI that checks that a stub package marked as complete can't have any incomplete (unannotated or using Incomplete) fields. Alternatively, we could add a disabled-by-default check to flake8-pyi and use that instead.
  3. At a later date, we could also add a PR CI script that add a PR comment warning a user if the amount of incomplete increases due to a PR.

(See also https://alexwaygood.github.io/typeshed-stats/.)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the stub package metadata in METADATA.toml and the current CI checks that use pyright's stricter configuration. Compare implementing a custom CI script with the proposed disabled-by-default flake8-pyi check. Done means complete packages cannot contain unannotated fields or fields using _typeshed.Incomplete, with the incomplete marker defaulting to false.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ci-cd, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.