lablup / lablup/backend.ai

Extend plug-in architecture

Open
#143 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

- Plugin base interfaces (a plugin may provide multiple interfaces):\* **Web application**: provides additional API endpoints
- **Auth provider**: provides an additional authentication/authorization scheme
- **Scaling driver**: provides an auto-scaling driver
- **CLI command**: provides additional manager commands
- **DB guest**: declares additional DB tables
- **Kernel tracer**: inserts hooks in the kernel lifecycle
- others to come

- [x] Auto-discovery of installed plugins (like pytest)\* Plugins should provide its metadata: name, version, available interfaces, and its version.

- [ ] Plugin inventory DB table for the manager\* Includes plugin metadata, (discovered) import path, currently installed version, when (de)activated last time.
- Additional fields:\* `config`: `json "{}"`
- `scope`: `string "global"`In the future, we may support scaling-group-specific plugins.

- [ ] Manager commands to activate/deactivate plugins and listing them\* Current draft "BACKEND_EXTENSIONS" will be dropped if this is implemented.
- The changes are applied upon next restart of the server.\* DB table creation/drops also happen on the next restart.

- Live reload is future work.

- [ ] Public framework APIs for the plugin authors\* Access to intrinsic DB tables/models
- Access to the configuration
- Interface-specific additions

- [ ] **Auth provider plugin**\* The base interface defines `authenticate()`.
- Target implementations: File-based, LDAP/AD-based, SAML-based

- [ ] **Web application plugin**\* The base interface is `aiohttp.web.Application`.
- The plugin routes are added under `"/ext/{plugin-id}/"`.

- [ ] **Scaling driver plugin**\* Delegated to lablup/backend.ai-manager#73.

- [ ] **CLI command plugin**\* The base interface defines signature for adding custom commands.
- Utilize `register_command` like intrinsic manager commands
- Could we seamlessly support auto-completion like lablup/backend.ai-client-py#24 ?

- [ ] **DB guest plugin** (#666)\* The base interface defines signatures for creation, upgrades, and deletion of prefixed DB tables
- On activation, the db tables are created.
- On deactivation, the admin is asked if to drop the tables or keep them at rest.
- The upgrade is automatically performed via the plugin. The plugin must implement its own way to migrate its table schemas.
- Plugin databases are _not_ part of the admin GraphQL interface. The plugin should provide its own API endpoint via web application interface if such functionality is required.

- [ ] **Kernel tracer plugin**\* The base interface defines hook methods that are called upon kernel/session-related events (e.g., creation and termination)
- It can also configure hooks to intercept execution requests and inspect/modify the requested codes.

JIRA Issue: BA-336

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.