bazel-contrib / bazel-contrib/rules_python

Example of using a "src" dir with gazelle

Open
#1,775 2 comments 0 reactions 0 assignees View on GitHub
gazelle type: documentation
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
14h 49m
Merged PRs (30d)
81

Description

N.B.: This is half "example request", half "how do I..." question.

# πŸš€ feature/example request

### Relevant Rules

+ py_*
+ gazelle

### Description

The [Python Packaging User Guide recommends using a `src` dir with tests outside of the package](https://packaging.python.org/en/latest/tutorials/packaging-projects/) (so they aren't shipped with the distribution/wheel), like so:

```
packaging_tutorial/
β”œβ”€β”€ LICENSE
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ README.md
β”œβ”€β”€ src/
β”‚ └── mypackage/
β”‚ β”œβ”€β”€ __init__.py
β”‚ └── foo.py
└── tests/
β”œβ”€β”€ __init__.py
└── test_foo.py
```

[`pytest` also recommends this](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#tests-outside-application-code).

However, none of the [examples](https://github.com/bazelbuild/rules_python/tree/main/examples) describe such a use case[^1].

[^1]: **Note:** the [bzlmod example](https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod) appears to do something similar with the `libs/my_lib` dir, but it's not quite the same because `libs/my_lib` doesn't need to be pip-installed to run tests. IMO the example also does too much, but that's a separate topic πŸ™ƒ.

Critically, one major aspect of the above dir structure is that the _project must be pip-installed[^2][^3] before tests can be run_ because `test_foo.py` looks like:

```python
import unittest

from mypackage import foo # here's the problem. Note that it's not `from src.mypackage import foo`

class TestFoo(unittest.testcase):
def test_add(self) -> None:
self.assertEqual(foo.add(1, 1), 2)
```

[^2]: typically as an editable package `pip install -e .`, but a non-editable install also works.
[^3]: Really all that's needed is `.../packaging_tutorial/src` to be in `PYTHONPATH`.

In addition, the documentation for `gazelle` is lacking and I haven't been able to figure out a way to get gazelle to work with a `src` dir.

#### Notes:

+ I think that https://github.com/bazelbuild/bazel/issues/6903 is similar.
+ I'm not asking for bazel to be able to do editable installs
+ as said in https://github.com/bazelbuild/rules_python/issues/434#issuecomment-1173007373, doing so prevents hermeticity

### Describe the solution you'd like

What I'd like to see is a new example added that showcases how to configure bazel and gazelle to work with a `src` dir.

In fact, I've [already got a repo for it](https://github.com/dougthor42/bazel-python-src-tests-example) that we can use as a starting point. General `bazel build|test|run` works, ~~but I am [still struggling with gazelle](https://github.com/dougthor42/bazel-python-src-tests-example/pull/3). **I'd be more than happy to build the example, but I'll need help doing so.**~~ [Edit 2024-04-11: With recent updates to gazelle, things are now working 😁]

The example would have the following structure (names are just suggestions, of course):

```
examples/src_dir_with_separate_tests/
β”œβ”€β”€ BUILD
β”œβ”€β”€ MODULE.bazel
β”œβ”€β”€ README.md
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ requirements.in
β”œβ”€β”€ src
β”‚   └── mypackage
β”‚      β”œβ”€β”€ BUILD
β”‚      β”œβ”€β”€ __init__.py
β”‚      β”œβ”€β”€ foo.py
β”‚      └── subpackage
β”‚      β”œβ”€β”€ BUILD
β”‚      β”œβ”€β”€ __init__.py
β”‚      └── subfoo.py
└── tests
β”œβ”€β”€ BUILD
β”œβ”€β”€ __init__.py
β”œβ”€β”€ subpackage
β”‚   β”œβ”€β”€ BUILD
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── test_subfoo.py
└── test_foo.py
```

### Describe alternatives you've considered

I tried looking for other examples on the web, but either my google-fu is failing me or there aren't any 😞.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.