Add more integration tests
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
First, read [the official integration testing guide](http://client-py.docs.backend.ai/en/latest/gsg/testing.html#integration-tests). Let's realize it.
Some of below functionalities are already covered in the test cases of the manager, but we need to test also API's input validation and matching of the client-side and server-side implementation.
- [x] Kernel integration tests (`tests/test_kernel.py`)\* Use this a starting point and reference. (It also may require some updates though)
- [ ] Admin integration tests (`tests/test_admin.py`) -> **needs updates!**\* [x] Create, delete, modify domains: use `testing-XXXX` format (with resource limits)
- [ ] Create, delete, modify groups (with resource limits)
- [ ] Create, delete, modify keypairs and users in different domains and groups (with resource limits)
- [ ] Check if the current user's resource limit reflects the domain/group limits as well if configured.
- [ ] Advanced kernel integration tests (`tests/test_kernel.py`)\* [ ] As a fixture, you need to create a domain, a group, and a user who belongs to them.
- [ ] Execute with custom environment variables using the above user
- [ ] Execute via websockets (stream APIs) using the above user
- [ ] Execute with the batch mode's build/clean commands, including cases with both satisfying and exceeding the batch-mode API's file number/size limits
- [ ] Activate service ports after creating kernels, and check if they give valid responses (e.g., some HTML codes for web-based container services like Jupyter)
- Group/domain segregation\* [ ] As a fixture, you need to create two or more domain/group/user sets.
- [ ] Check if running kernels for a specific domain and group is not listed in other user in a different domain and group.
- Resource limits\* [ ] Execute with different resource limits and check/hit those limits (e.g., check the number of CPU cores, check if an OOM event (forced termination) occurs)
- [ ] Try to create kernels whose creation configs exceed the configured domain, group, user (keypair) resource limits and check if the limits are enforced as expected.
- [ ] VFolder integration tests (`tests/test_vfolder.py`)\* [x] Create and delete personal vfolders, check them using the listing API
- [ ] Hit the vfolder limit of the keypair resource policy by creating too many vfolders
- [ ] Upload/download small (10 MiB) and large (10 GiB) random-generated files with hash checks
- Group vfolders\* [ ] As a fixture, you need to create two or more groups and corresponding domain admin users.
- [ ] Create and delete group vfolders, check them using the listing API
- [ ] Check if users who belong to only a specific group cannot list & access another group's vfolders.
- Domain segregation\* [ ] As a fixture, you need to create two or more domain/group/user sets.
- [ ] Check if vfolders for a specific domain cannot send/receive invitations to/from users in a different domain and group.
JIRA Issue: BA-329
Contributor guide
Assessment
This issue has not been assessed yet.