microsoft / microsoft/pyright

Pyright fails on files that are too large

Open
#10,910 9 comments 0 reactions 0 assignees View on GitHub
addressed in next version bug
Dominant language
Python
Stars
15.6k
Forks
1.8k
Avg merge
12h 13m
Merged PRs (30d)
52

Description

Pyright started failing on our repo a few weeks ago, providing incorrect hints in VS Code, though mypy was successfully passing. We noticed that the issue was triggered when we bumped one of the dependencies, which boils down to one very large (~90k line), autogenerated Python file. I then turned this into a min repro with the `pyright` CLI.

**Describe the bug**

With `nominal-api==0.802.0` installed from PyPI, the following script passes a `pyright` type check. With `nominal-api==0.803.0` (and above), the following script fails to type check. However, the code should pass in both cases.

**Code or Screenshots**

```python asdf.py
from nominal_api.module import ModuleApplication, ModuleRef

def f(s0: str, s1: str, s2: str) -> str:
return s0 + s1 + s2

mod = ModuleApplication("a", "b", "c", ModuleRef("d"), "e")
f(mod.rid, mod.asset_rid, mod.module.rid)
```

To repro, name the above script `asdf.py`, then:

```sh
% uv add pyright nominal-api==0.802.0
% uv run pyright asdf.py --verbose
% uv add nominal-api==0.803.0
% uv run pyright asdf.py --verbose
```

Note that the diff from the `nominal-api` versions 0.802.0 to 0.803.0 is pretty small compared to the file size, and ultimately inconsequential:

```diff 802_impl.py 803_impl.py```

```
14800a14801,14805
>
>
> class module_LatestVersionStrategy(ConjureBeanType):
> """This strategy refers to the latest version of the module.
> """
14801a14807,14810
> @builtins.classmethod
> def _fields(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> }
14802a14812,14820
> __slots__: List[str] = []
>
>
>
> module_LatestVersionStrategy.__name__ = "LatestVersionStrategy"
> module_LatestVersionStrategy.__qualname__ = "LatestVersionStrategy"
> module_LatestVersionStrategy.__module__ = "nominal_api.module"
>
>
15492a15511,15624
>
>
> class module_PinnedVersionStrategy(ConjureBeanType):
> """This strategy refers to a specific version of the module.
> """
>
> @builtins.classmethod
> def _fields(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> 'version': ConjureFieldDefinition('version', module_ModuleVersion)
> }
>
> __slots__: List[str] = ['_version']
>
> def __init__(self, version: str) -> None:
> self._version = version
>
> @builtins.property
> def version(self) -> str:
> return self._version
>
>
> module_PinnedVersionStrategy.__name__ = "PinnedVersionStrategy"
> module_PinnedVersionStrategy.__qualname__ = "PinnedVersionStrategy"
> module_PinnedVersionStrategy.__module__ = "nominal_api.module"
>
>
> class module_RequestModuleNameRef(ConjureBeanType):
> """This is used to refer to modules in requests by name.
> """
>
> @builtins.classmethod
> def _fields(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> 'name': ConjureFieldDefinition('name', str),
> 'version_strategy': ConjureFieldDefinition('versionStrategy', module_VersionStrategy)
> }
>
> __slots__: List[str] = ['_name', '_version_strategy']
>
> def __init__(self, name: str, version_strategy: "module_VersionStrategy") -> None:
> self._name = name
> self._version_strategy = version_strategy
>
> @builtins.property
> def name(self) -> str:
> return self._name
>
> @builtins.property
> def version_strategy(self) -> "module_VersionStrategy":
> return self._version_strategy
>
>
> module_RequestModuleNameRef.__name__ = "RequestModuleNameRef"
> module_RequestModuleNameRef.__qualname__ = "RequestModuleNameRef"
> module_RequestModuleNameRef.__module__ = "nominal_api.module"
>
>
> class module_RequestModuleRef(ConjureUnionType):
> """Request reference to a module. This is used to refer to modules in requests.
> """
> _name: Optional["module_RequestModuleNameRef"] = None
>
> @builtins.classmethod
> def _options(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> 'name': ConjureFieldDefinition('name', module_RequestModuleNameRef)
> }
>
> def __init__(
> self,
> name: Optional["module_RequestModuleNameRef"] = None,
> type_of_union: Optional[str] = None
> ) -> None:
> if type_of_union is None:
> if (name is not None) != 1:
> raise ValueError('a union must contain a single member')
>
> if name is not None:
> self._name = name
> self._type = 'name'
>
> elif type_of_union == 'name':
> if name is None:
> raise ValueError('a union value must not be None')
> self._name = name
> self._type = 'name'
>
> @builtins.property
> def name(self) -> Optional["module_RequestModuleNameRef"]:
> return self._name
>
> def accept(self, visitor) -> Any:
> if not isinstance(visitor, module_RequestModuleRefVisitor):
> raise ValueError('{} is not an instance of module_RequestModuleRefVisitor'.format(visitor.__class__.__name__))
> if self._type == 'name' and self.name is not None:
> return visitor._name(self.name)
>
>
> module_RequestModuleRef.__name__ = "RequestModuleRef"
> module_RequestModuleRef.__qualname__ = "RequestModuleRef"
> module_RequestModuleRef.__module__ = "nominal_api.module"
>
>
> class module_RequestModuleRefVisitor:
>
> @abstractmethod
> def _name(self, name: "module_RequestModuleNameRef") -> Any:
> pass
>
>
> module_RequestModuleRefVisitor.__name__ = "RequestModuleRefVisitor"
> module_RequestModuleRefVisitor.__qualname__ = "RequestModuleRefVisitor"
> module_RequestModuleRefVisitor.__module__ = "nominal_api.module"
16029,16050d16160
<
<
< class module_SemanticVersion(ConjureBeanType):
<
< @builtins.classmethod
< def _fields(cls) -> Dict[str, ConjureFieldDefinition]:
< return {
< 'major': ConjureFieldDefinition('major', int),
< 'minor': ConjureFieldDefinition('minor', int),
< 'patch': ConjureFieldDefinition('patch', int)
< }
<
< __slots__: List[str] = ['_major', '_minor', '_patch']
<
< def __init__(self, major: int, minor: int, patch: int) -> None:
< self._major = major
< self._minor = minor
< self._patch = patch
<
< @builtins.property
< def major(self) -> int:
< return self._major
16052,16054d16161
< @builtins.property
< def minor(self) -> int:
< return self._minor
16056,16065d16162
< @builtins.property
< def patch(self) -> int:
< return self._patch
<
<
< module_SemanticVersion.__name__ = "SemanticVersion"
< module_SemanticVersion.__qualname__ = "SemanticVersion"
< module_SemanticVersion.__module__ = "nominal_api.module"
<
<
16188a16286,16297
>
>
> class module_VersionStrategy(ConjureUnionType):
> _pinned: Optional["module_PinnedVersionStrategy"] = None
> _latest: Optional["module_LatestVersionStrategy"] = None
>
> @builtins.classmethod
> def _options(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> 'pinned': ConjureFieldDefinition('pinned', module_PinnedVersionStrategy),
> 'latest': ConjureFieldDefinition('latest', module_LatestVersionStrategy)
> }
16189a16299,16307
> def __init__(
> self,
> pinned: Optional["module_PinnedVersionStrategy"] = None,
> latest: Optional["module_LatestVersionStrategy"] = None,
> type_of_union: Optional[str] = None
> ) -> None:
> if type_of_union is None:
> if (pinned is not None) + (latest is not None) != 1:
> raise ValueError('a union must contain a single member')
16190a16309,16364
> if pinned is not None:
> self._pinned = pinned
> self._type = 'pinned'
> if latest is not None:
> self._latest = latest
> self._type = 'latest'
>
> elif type_of_union == 'pinned':
> if pinned is None:
> raise ValueError('a union value must not be None')
> self._pinned = pinned
> self._type = 'pinned'
> elif type_of_union == 'latest':
> if latest is None:
> raise ValueError('a union value must not be None')
> self._latest = latest
> self._type = 'latest'
>
> @builtins.property
> def pinned(self) -> Optional["module_PinnedVersionStrategy"]:
> return self._pinned
>
> @builtins.property
> def latest(self) -> Optional["module_LatestVersionStrategy"]:
> return self._latest
>
> def accept(self, visitor) -> Any:
> if not isinstance(visitor, module_VersionStrategyVisitor):
> raise ValueError('{} is not an instance of module_VersionStrategyVisitor'.format(visitor.__class__.__name__))
> if self._type == 'pinned' and self.pinned is not None:
> return visitor._pinned(self.pinned)
> if self._type == 'latest' and self.latest is not None:
> return visitor._latest(self.latest)
>
>
> module_VersionStrategy.__name__ = "VersionStrategy"
> module_VersionStrategy.__qualname__ = "VersionStrategy"
> module_VersionStrategy.__module__ = "nominal_api.module"
>
>
> class module_VersionStrategyVisitor:
>
> @abstractmethod
> def _pinned(self, pinned: "module_PinnedVersionStrategy") -> Any:
> pass
>
> @abstractmethod
> def _latest(self, latest: "module_LatestVersionStrategy") -> Any:
> pass
>
>
> module_VersionStrategyVisitor.__name__ = "VersionStrategyVisitor"
> module_VersionStrategyVisitor.__qualname__ = "VersionStrategyVisitor"
> module_VersionStrategyVisitor.__module__ = "nominal_api.module"
>
>
16242,16243c16416
< """Returns the resolved module definitions for the given module-asset pairs. If any of modules have not been
< applied to their corresponding asset, this will throw.
---
> """Returns the resolved module definitions for the requested ModuleApplication.
16273a16447,16450
> def get_unresolved_module_definition(self, auth_header: str, module_ref: "module_RequestModuleNameRef") -> "module_internal_ModuleComputeDefinition":
> """Returns the module definition for the given module reference.
> """
> _conjure_encoder = ConjureEncoder()
16274a16452,16479
> _headers: Dict[str, Any] = {
> 'Accept': 'application/json',
> 'Content-Type': 'application/json',
> 'Authorization': auth_header,
> }
>
> _params: Dict[str, Any] = {
> }
>
> _path_params: Dict[str, str] = {
> }
>
> _json: Any = _conjure_encoder.default(module_ref)
>
> _path = '/internal/scout/v2/module/module/get'
> _path = _path.format(**_path_params)
>
> _response: Response = self._request(
> 'POST',
> self._uri + _path,
> params=_params,
> headers=_headers,
> json=_json)
>
> _decoder = ConjureDecoder()
> return _decoder.decode(_response.json(), module_internal_ModuleComputeDefinition, self._return_none_for_unknown_union_types)
>
>
16303c16508
< class module_internal_ResolvedModuleVersionDefinition(ConjureBeanType):
---
> class module_internal_ModuleComputeDefinition(ConjureBeanType):
16308,16309c16513,16515
< 'module_application_rid': ConjureFieldDefinition('moduleApplicationRid', modules_api_ModuleApplicationRid),
< 'resolved_parameters': ConjureFieldDefinition('resolvedParameters', List[module_ModuleVariable]),
---
> 'module_name': ConjureFieldDefinition('moduleName', str),
> 'module_rid': ConjureFieldDefinition('moduleRid', modules_api_ModuleRid),
> 'version': ConjureFieldDefinition('version', module_ModuleVersion),
16314c16520
< __slots__: List[str] = ['_module_application_rid', '_resolved_parameters', '_default_variables', '_functions']
---
> __slots__: List[str] = ['_module_name', '_module_rid', '_version', '_default_variables', '_functions']
16316,16318c16522,16525
< def __init__(self, default_variables: List["module_ModuleVariable"], functions: List["module_Function"], module_application_rid: str, resolved_parameters: List["module_ModuleVariable"]) -> None:
< self._module_application_rid = module_application_rid
< self._resolved_parameters = resolved_parameters
---
> def __init__(self, default_variables: List["module_ModuleVariable"], functions: List["module_Function"], module_name: str, module_rid: str, version: str) -> None:
> self._module_name = module_name
> self._module_rid = module_rid
> self._version = version
16323,16324c16530,16531
< def module_application_rid(self) -> str:
< return self._module_application_rid
---
> def module_name(self) -> str:
> return self._module_name
16327,16328c16534,16539
< def resolved_parameters(self) -> List["module_ModuleVariable"]:
< return self._resolved_parameters
---
> def module_rid(self) -> str:
> return self._module_rid
>
> @builtins.property
> def version(self) -> str:
> return self._version
16336a16548,16565
>
>
> module_internal_ModuleComputeDefinition.__name__ = "ModuleComputeDefinition"
> module_internal_ModuleComputeDefinition.__qualname__ = "ModuleComputeDefinition"
> module_internal_ModuleComputeDefinition.__module__ = "nominal_api.module_internal"
>
>
> class module_internal_ResolvedModuleVersionDefinition(ConjureBeanType):
>
> @builtins.classmethod
> def _fields(cls) -> Dict[str, ConjureFieldDefinition]:
> return {
> 'module_application_rid': ConjureFieldDefinition('moduleApplicationRid', modules_api_ModuleApplicationRid),
> 'resolved_parameters': ConjureFieldDefinition('resolvedParameters', List[module_ModuleVariable]),
> 'module_definition': ConjureFieldDefinition('moduleDefinition', module_internal_ModuleComputeDefinition)
> }
>
> __slots__: List[str] = ['_module_application_rid', '_resolved_parameters', '_module_definition']
16337a16567,16570
> def __init__(self, module_application_rid: str, module_definition: "module_internal_ModuleComputeDefinition", resolved_parameters: List["module_ModuleVariable"]) -> None:
> self._module_application_rid = module_application_rid
> self._resolved_parameters = resolved_parameters
> self._module_definition = module_definition
16338a16572,16584
> @builtins.property
> def module_application_rid(self) -> str:
> return self._module_application_rid
>
> @builtins.property
> def resolved_parameters(self) -> List["module_ModuleVariable"]:
> return self._resolved_parameters
>
> @builtins.property
> def module_definition(self) -> "module_internal_ModuleComputeDefinition":
> return self._module_definition
>
>
88894a89141,89142
> module_ModuleVersion = str
>
```

**VS Code extension or command-line**

```sh
% uv run pyright --version
pyright 1.1.405
```

**Final notes**

I'm not quite sure what `pyright` limit we're hitting, but whatever it is, it would be nice to have some configuration to tweak it a bit. As it stands we can no longer use `pyright` on our repo, and have to disable it and instead get type hinting during coding with mypy. Pyright fails all over the place here, it starts ignoring `@property`, treating them instead as methods, it totally misses some types altogether. In general, against the `nominal-client` repo, there's 320 pyright errors reported after upgrading from `nominal-api` 0.802.0 -> 0.803.0 despite `nominal-client` not utilizing any code that is in the diff, and none of it being deleted, either.

For reference, the autogened code is from the [conjure](https://github.com/palantir/conjure) API ecosystem. While the code is _long_, it's extremely simple. It's low cyclomatic complexity, it's just classes and methods, and the methods are trivial in implementation. You can view the Python code from the sdist on PyPI: https://pypi.org/project/nominal-api/

Contributor guide

Open the contributing guide

Research direction

Start with the minimal asdf.py example and run the listed uv and pyright CLI commands against nominal-api 0.802.0 and 0.803.0 using --verbose. Trace how Pyright handles the very large autogenerated Python module and confirm the example type-checks consistently across the dependency versions without incorrect hints.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
devtools
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
66/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.