Abstract test case
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 279
- Avg merge
- 2d 1h
- Merged PRs (30d)
- 1
Description
Sometimes we have some interface and we would like to write a generic test suite for it, so that concrete implementations just reuse it.
The problem with Python's `unittest` (and transitively `absltest`) is that the `TestCase` class serves as a "marker" telling the test executor that it should be executed. This is unfortunate, because abstract test suites should not run. Consider this example:
```python
class FooTest(absltest.TestCase):
@abstractmethod
def new_foo(self) -> Foo:
pass
def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")
class QuuxTest(FooTest):
def new_foo(self) -> Foo:
return Quux()
class NorfTest(FooTest):
def new_foo(self) -> Foo:
return Norf()
```
Here, the test executor will instantiate `FooTest`, `QuuxTest` and `NorfTest`. However, `FooTest` is abstract and it is not possible to create an instance of it. There are three workarounds for this that I am aware of.
The first one is to configure the test executor to ignore specific test classes or prefixes (possible in pytest). However, this is awkward and requires modifying external configuration files.
The second one is described [here](https://stackoverflow.com/a/22836015). Basically, we `del` the base class once child classes are defined. This feels *very* wrong and works only if all the concrete test cases are defined in the same module.
The last one is to use a mixin approach. Instead of making `FooTest` derive from `absltest.TestCase`, we "mark" only concrete classes with it:
```python
class FooTest:
@abstractmethod
def new_foo(self) -> Foo:
pass
def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")
class QuuxTest(FooTest, absltest.TestCase):
def new_foo(self) -> Foo:
return Quux()
class NorfTest(FooTest, absltest.TestCase):
def new_foo(self) -> Foo:
return Norf()
```
The problem here is that it doesn't work with static type checkers: `FooTest` now doesn't inherit from `TestCase` but uses methods like `self.assertEqual`.
This last solution seems like the only "correct" one except for the mentioned issue. Instead, Abseil could define an abstract test class and make the normal test case class implement it:
```python
class AbstractTestCase(ABC):
@abstractmethod
def assertEqual(self, this, that):
...
...
class TestCase(AbstractTestCase):
...
```
Then, it would be possible to inherit from `absltest.AbstractTestCase` in the abstract test case, making the type checker happy:
```python
class FooTest(absltest.AbstractTestCase):
@abstractmethod
def new_foo(self) -> Foo:
pass
def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")
class QuuxTest(FooTest, absltest.TestCase):
def new_foo(self) -> Foo:
return Quux()
class NorfTest(FooTest, absltest.TestCase):
def new_foo(self) -> Foo:
return Norf()
```
Contributor guide
Assessment
This issue has not been assessed yet.