googleapis / googleapis/gapic-showcase

Test Isolation

Open
#82 1 comment 0 reactions 0 assignees View on GitHub
priority: p2 type: cleanup
Dominant language
Go
Stars
183
Forks
55
Avg merge
1d 1h
Merged PRs (30d)
8

Description

I started writing a CRUD test today:

```python
def test_crud_with_request_dict(identity):
assert len(identity.list_users().users) == 0
try:
user = identity.create_user({
'display_name': 'Guido van Rossum',
'email': 'guido@guido.fake',
})
assert user.display_name == 'Guido van Rossum'
assert user.email == 'guido@guido.fake'
assert len(identity.list_users().users) == 1
assert identity.get_user({
'name': user.name,
}).display_name == 'Guido van Rossum'
finally:
identity.delete_user({'name': user.name})
```

This made me realize that we may want some way of supporting Showcase-enforced test isolation. I need it to be the case that if something goes wrong in _this_ test, that it does not cascade to a bunch of other tests. Basically a "okay, this test is over, drop any data changes it made in the Identity or Messaging services".

That said, this warrants discussion. We may not want to do this, and there are other options:

* Making tests not care about the absolute state of the database (e.g. test that `len` increased by one, not that it is equal to one).
* Make tests care about the absolute state of the database, but skip the test if a beginning-state guard clause fails. This will cause situations where one test failure will lead to a host of other tests being skipped. That is okay with me actually -- it will be easy to isolate the failure.

That said, some kind of "transaction with rollback" is probably the best thing.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.