protocolbuffers / protocolbuffers/protobuf

[Python] Generated type annotations do not pass mypy strict

Open Beginner friendly
#29,528 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature request untriaged
Dominant language
C++
Stars
72k
Forks
16.3k
Avg merge
1d 17h
Merged PRs (30d)
140

Description

Hi! Here is a request to improve pyi Python type annotations generated.
Using the latest protoc version, and assuming this proto file:

syntax = "proto3";

message X {
  string s = 1;
}

message M {
  repeated X xs = 1;
}

After generating the code and pyi file:

$ protoc repro.proto --python_out=. --pyi_out=.

Here is the generated pyi file:

from google.protobuf.internal import containers as _containers
from google.protobuf import descriptor as _descriptor
from google.protobuf import message as _message
from collections.abc import Iterable as _Iterable, Mapping as _Mapping
from typing import ClassVar as _ClassVar, Optional as _Optional, Union as _Union

DESCRIPTOR: _descriptor.FileDescriptor

class X(_message.Message):
    __slots__ = ("s",)
    S_FIELD_NUMBER: _ClassVar[int]
    s: str
    def __init__(self, s: _Optional[str] = ...) -> None: ...

class M(_message.Message):
    __slots__ = ("xs",)
    XS_FIELD_NUMBER: _ClassVar[int]
    xs: _containers.RepeatedCompositeFieldContainer[X]
    def __init__(self, xs: _Optional[_Iterable[_Union[X, _Mapping]]] = ...) -> None: ...

mypy (the standard Python type checker) raises a type-arg issue in strict mode:

# python -m mypy repro_pb2.pyi  --strict, needs types-protobuf
repro_pb2.pyi:19: error: Missing type arguments for generic type "Mapping"  [type-arg]

In this example, the _Mapping is indeed lacking type arguments, and could be typed as Mapping[str, str].
AFAICT this is the only issue I encountered of such type. Fixing it could help the adoption of static typing of projects relying on generated protobuf code.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the issue with the shown repro.proto and protoc repro.proto --python_out=. --pyi_out=. command, then run mypy in strict mode on the generated .pyi file. Trace the pyi generation path responsible for the _Mapping annotation and make the generated annotation fully parameterized; done means the example passes strict mypy without the type-arg error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
developer-experience
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.