bazel-contrib / bazel-contrib/rules_python

Example of using a "src" dir with gazelle

Open
#1,775 2 comments 0 reactions 0 assignees View on GitHub
gazelle type: documentation
Dominant language
Starlark
Stars
688
Forks
721
Avg merge
15h 7m
Merged PRs (30d)
76

Description

N.B.: This is half "example request", half "how do I..." question.

# πŸš€ feature/example request

### Relevant Rules

+ py_*
+ gazelle

### Description

The [Python Packaging User Guide recommends using a `src` dir with tests outside of the package](https://packaging.python.org/en/latest/tutorials/packaging-projects/) (so they aren't shipped with the distribution/wheel), like so:

```
packaging_tutorial/
β”œβ”€β”€ LICENSE
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ README.md
β”œβ”€β”€ src/
β”‚ └── mypackage/
β”‚ β”œβ”€β”€ __init__.py
β”‚ └── foo.py
└── tests/
β”œβ”€β”€ __init__.py
└── test_foo.py
```

[`pytest` also recommends this](https://docs.pytest.org/en/7.1.x/explanation/goodpractices.html#tests-outside-application-code).

However, none of the [examples](https://github.com/bazelbuild/rules_python/tree/main/examples) describe such a use case[^1].

[^1]: **Note:** the [bzlmod example](https://github.com/bazelbuild/rules_python/tree/main/examples/bzlmod) appears to do something similar with the `libs/my_lib` dir, but it's not quite the same because `libs/my_lib` doesn't need to be pip-installed to run tests. IMO the example also does too much, but that's a separate topic πŸ™ƒ.

Critically, one major aspect of the above dir structure is that the _project must be pip-installed[^2][^3] before tests can be run_ because `test_foo.py` looks like:

```python
import unittest

from mypackage import foo # here's the problem. Note that it's not `from src.mypackage import foo`

class TestFoo(unittest.testcase):
def test_add(self) -> None:
self.assertEqual(foo.add(1, 1), 2)
```

[^2]: typically as an editable package `pip install -e .`, but a non-editable install also works.
[^3]: Really all that's needed is `.../packaging_tutorial/src` to be in `PYTHONPATH`.

In addition, the documentation for `gazelle` is lacking and I haven't been able to figure out a way to get gazelle to work with a `src` dir.

#### Notes:

+ I think that https://github.com/bazelbuild/bazel/issues/6903 is similar.
+ I'm not asking for bazel to be able to do editable installs
+ as said in https://github.com/bazelbuild/rules_python/issues/434#issuecomment-1173007373, doing so prevents hermeticity

### Describe the solution you'd like

What I'd like to see is a new example added that showcases how to configure bazel and gazelle to work with a `src` dir.

In fact, I've [already got a repo for it](https://github.com/dougthor42/bazel-python-src-tests-example) that we can use as a starting point. General `bazel build|test|run` works, ~~but I am [still struggling with gazelle](https://github.com/dougthor42/bazel-python-src-tests-example/pull/3). **I'd be more than happy to build the example, but I'll need help doing so.**~~ [Edit 2024-04-11: With recent updates to gazelle, things are now working 😁]

The example would have the following structure (names are just suggestions, of course):

```
examples/src_dir_with_separate_tests/
β”œβ”€β”€ BUILD
β”œβ”€β”€ MODULE.bazel
β”œβ”€β”€ README.md
β”œβ”€β”€ pyproject.toml
β”œβ”€β”€ requirements.in
β”œβ”€β”€ src
β”‚   └── mypackage
β”‚      β”œβ”€β”€ BUILD
β”‚      β”œβ”€β”€ __init__.py
β”‚      β”œβ”€β”€ foo.py
β”‚      └── subpackage
β”‚      β”œβ”€β”€ BUILD
β”‚      β”œβ”€β”€ __init__.py
β”‚      └── subfoo.py
└── tests
β”œβ”€β”€ BUILD
β”œβ”€β”€ __init__.py
β”œβ”€β”€ subpackage
β”‚   β”œβ”€β”€ BUILD
β”‚   β”œβ”€β”€ __init__.py
β”‚   └── test_subfoo.py
└── test_foo.py
```

### Describe alternatives you've considered

I tried looking for other examples on the web, but either my google-fu is failing me or there aren't any 😞.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the existing examples/ directory and the linked bazel-python-src-tests-example repository, especially its current Gazelle setup. Done means adding a focused examples/src_dir_with_separate_tests example with the proposed Bazel, module, Python package, test, and README structure, and documenting that bazel build, test, and run work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.