bazel-contrib / bazel-contrib/rules_python
Example of using a "src" dir with gazelle
- Dominant language
- Starlark
- Stars
- 688
- Forks
- 721
- Avg merge
- 15h 7m
- Merged PRs (30d)
- 76
Description
N.B.: This is half "example request", half "how do I..." question.
# π feature/example request
### Relevant Rules
+ py_*
+ gazelle
### Description
The [Python Packaging User Guide recommends using a `src` dir with tests outside of the package](https://packaging.python.org/en/latest/tutorials/packaging-projects/) (so they aren't shipped with the distribution/wheel), like so:
```
packaging_tutorial/
βββ LICENSE
βββ pyproject.toml
βββ README.md
βββ src/
β βββ mypackage/
β βββ __init__.py
β βββ foo.py
βββ tests/
βββ __init__.py
βββ test_foo.py
```
[`pytest` also recommends this](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#tests-outside-application-code).
However, none of the [examples](https://github.com/bazelbuild/rules_python/tree/main/examples) describe such a use case[^1].
[^1]: **Note:** the [bzlmod example](https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod) appears to do something similar with the `libs/my_lib` dir, but it's not quite the same because `libs/my_lib` doesn't need to be pip-installed to run tests. IMO the example also does too much, but that's a separate topic π.
Critically, one major aspect of the above dir structure is that the _project must be pip-installed[^2][^3] before tests can be run_ because `test_foo.py` looks like:
```python
import unittest
from mypackage import foo # here's the problem. Note that it's not `from src.mypackage import foo`
class TestFoo(unittest.testcase):
def test_add(self) -> None:
self.assertEqual(foo.add(1, 1), 2)
```
[^2]: typically as an editable package `pip install -e .`, but a non-editable install also works.
[^3]: Really all that's needed is `.../packaging_tutorial/src` to be in `PYTHONPATH`.
In addition, the documentation for `gazelle` is lacking and I haven't been able to figure out a way to get gazelle to work with a `src` dir.
#### Notes:
+ I think that https://github.com/bazelbuild/bazel/issues/6903 is similar.
+ I'm not asking for bazel to be able to do editable installs
+ as said in https://github.com/bazelbuild/rules_python/issues/434#issuecomment-1173007373, doing so prevents hermeticity
### Describe the solution you'd like
What I'd like to see is a new example added that showcases how to configure bazel and gazelle to work with a `src` dir.
In fact, I've [already got a repo for it](https://github.com/dougthor42/bazel-python-src-tests-example) that we can use as a starting point. General `bazel build|test|run` works, ~~but I am [still struggling with gazelle](https://github.com/dougthor42/bazel-python-src-tests-example/pull/3). **I'd be more than happy to build the example, but I'll need help doing so.**~~ [Edit 2024-04-11: With recent updates to gazelle, things are now working π]
The example would have the following structure (names are just suggestions, of course):
```
examples/src_dir_with_separate_tests/
βββ BUILD
βββ MODULE.bazel
βββ README.md
βββ pyproject.toml
βββ requirements.in
βββ src
β βββ mypackage
β βββ BUILD
β βββ __init__.py
β βββ foo.py
β βββ subpackage
β βββ BUILD
β βββ __init__.py
β βββ subfoo.py
βββ tests
βββ BUILD
βββ __init__.py
βββ subpackage
β βββ BUILD
β βββ __init__.py
β βββ test_subfoo.py
βββ test_foo.py
```
### Describe alternatives you've considered
I tried looking for other examples on the web, but either my google-fu is failing me or there aren't any π.
Contributor guide
Research direction
Start by reviewing the existing examples/ directory and the linked bazel-python-src-tests-example repository, especially its current Gazelle setup. Done means adding a focused examples/src_dir_with_separate_tests example with the proposed Bazel, module, Python package, test, and README structure, and documenting that bazel build, test, and run work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- build-system, documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100