abseil / abseil/abseil-py

Abstract test case

Open
#166 6 comments 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
279
Avg merge
2d 1h
Merged PRs (30d)
1

Description

Sometimes we have some interface and we would like to write a generic test suite for it, so that concrete implementations just reuse it.

The problem with Python's `unittest` (and transitively `absltest`) is that the `TestCase` class serves as a "marker" telling the test executor that it should be executed. This is unfortunate, because abstract test suites should not run. Consider this example:

```python
class FooTest(absltest.TestCase):

@abstractmethod
def new_foo(self) -> Foo:
pass

def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")

class QuuxTest(FooTest):

def new_foo(self) -> Foo:
return Quux()

class NorfTest(FooTest):

def new_foo(self) -> Foo:
return Norf()
```

Here, the test executor will instantiate `FooTest`, `QuuxTest` and `NorfTest`. However, `FooTest` is abstract and it is not possible to create an instance of it. There are three workarounds for this that I am aware of.

The first one is to configure the test executor to ignore specific test classes or prefixes (possible in pytest). However, this is awkward and requires modifying external configuration files.

The second one is described [here](https://stackoverflow.com/a/22836015). Basically, we `del` the base class once child classes are defined. This feels *very* wrong and works only if all the concrete test cases are defined in the same module.

The last one is to use a mixin approach. Instead of making `FooTest` derive from `absltest.TestCase`, we "mark" only concrete classes with it:

```python
class FooTest:

@abstractmethod
def new_foo(self) -> Foo:
pass

def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")

class QuuxTest(FooTest, absltest.TestCase):

def new_foo(self) -> Foo:
return Quux()

class NorfTest(FooTest, absltest.TestCase):

def new_foo(self) -> Foo:
return Norf()
```

The problem here is that it doesn't work with static type checkers: `FooTest` now doesn't inherit from `TestCase` but uses methods like `self.assertEqual`.

This last solution seems like the only "correct" one except for the mentioned issue. Instead, Abseil could define an abstract test class and make the normal test case class implement it:

```python
class AbstractTestCase(ABC):

@abstractmethod
def assertEqual(self, this, that):
...

...

class TestCase(AbstractTestCase):
...
```

Then, it would be possible to inherit from `absltest.AbstractTestCase` in the abstract test case, making the type checker happy:

```python
class FooTest(absltest.AbstractTestCase):

@abstractmethod
def new_foo(self) -> Foo:
pass

def test_bar(self) -> None:
foo = self.new_foo()
self.assertEqual(foo.bar(), "bar")

class QuuxTest(FooTest, absltest.TestCase):

def new_foo(self) -> Foo:
return Quux()

class NorfTest(FooTest, absltest.TestCase):

def new_foo(self) -> Foo:
return Norf()
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.