Integrated tests for kernels
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
This issue replaces lablup/backend.ai-kernels#3 and represents the work in [lablup/backend.ai-kernel-tester](https://github.com/lablup/backend.ai-kernel-tester).
The goals:
- Execute realistic/real code samples (with minimized input data) by collecting actual error cases.
- Run them in a fully integrated setup: manager + agent + accelerator plugin.\* For now, we assume that the user of the test suite already has a working setup.
- Make it possible to reuse a code sample across multiple different kernels if the kernels contain the same set of dependency packages for the sample.
Future goals:
- Automate execution of this integration test suites upon every push to the `backend.ai-kernels` repository, only selectively by running tests related to the commit changes.
- Prepare a high-performance kernel image builder farm (Azure Pipelines? on-prem?)
- Make it possible to run the same tests using different version combinations of the manager/agent/accelerator plugins/Docker/OS, in an automated way. (Vagrant + Ansible?)\* It is not practical to run all possible combinations as CI for every push. For CI, just keep a few representative (stable-based) combinations, but let's allow customization of the combinations with automated setup.
┆Issue is synchronized with this [Asana task](https://app.asana.com/0/1159751085623729/1159756855629509) by [Unito](https://www.unito.io/learn-more)
```Java
### Tasks
- [ ] https://github.com/lablup/backend.ai/issues/1089
- [ ] https://github.com/lablup/backend.ai/issues/1088
```
JIRA Issue: BA-316
Contributor guide
Assessment
This issue has not been assessed yet.