Add `namespaceOverridePaths` configOption for fragmented package support
- Dominant language
- Python
- Stars
- 15.6k
- Forks
- 1.8k
- Avg merge
- 12h 13m
- Merged PRs (30d)
- 52
Description
This is a request to introduce a new configuration setting, namespaceOverridePaths, designed to align Pyright’s static analysis with the complex runtime import behaviors of modern build systems (like Bazel) and legacy Python namespace patterns.
### Problem
Currently, the ImportResolver strictly adheres to standard PEP 484/561 logic: if an __init__.py file is found, that directory is classified as a Regular Package. The search immediately terminates at that root, and the resolver ignores any subsequent entries in the sys.path.
While this is correct for standard environments, it creates significant "shadowing" issues in environments where a single logical package is physically fragmented across multiple directories.
The Shadowing Problem
1. **Bazel & rules_python**
Bazel constructs a virtual execroot (runfiles tree) by symlinking targets from various locations. A project might have:
```
src/pkg/core/__init__.py (Main Repo)
gen/pkg/core/__init__.py (Generated Code)
external/pip_dep/pkg/__init__.py (Third-party)
```
In the Bazel runtime, all three directories are added to the `PYTHONPATH`. However, after adding these directories to the `python.analyzer.extraPaths` if the ImportResolver finds the `__init__.py` in `src/pkg/core`, it stops. It shadows the generated protos and the third-party utils, leading to "Module not found" errors in the IDE that do not exist at runtime.
2. **Legacy pkgutil / pkg_resources Namespaces**
Many enterprise libraries (e.g., google-cloud-sdk, azure-namespace) still use legacy namespace declarations:
```python
__path__ = __import__('pkgutil').extend_path(__path__, __name__)
```
These libraries contain an `__init__.py` specifically to allow merging with other directories. Currently, the resolver treats these as terminal Regular Packages, effectively breaking the discovery of sub-packages distributed across multiple installed wheels.
### Solution
The proposed change is to introduces a configuration array that allows users to explicitly define transparent namespaces. When the resolver encounters a directory within these paths, it treats it as a namespace segment even if an __init__.py is present.
I have a draft pr https://github.com/microsoft/pyright/pull/11256 open that addresses this enhancement with a full implementation, schema updates, and comprehensive test coverage.
Contributor guide
Research direction
Start by reviewing the draft implementation in pull request #11256 and the ImportResolver behavior described here. Check its schema updates and comprehensive test coverage, then verify that namespaceOverridePaths lets configured directories merge despite __init__.py files without changing standard package resolution.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100