kestra-io / kestra-io/plugin-oci
[Plugin] OCI — Database (Autonomous Database & MySQL HeatWave)
- Dominant language
- Java
- Stars
- 0
- Forks
- 0
- Avg merge
- 18h 3m
- Merged PRs (30d)
- 1
Description
## Summary
The OCI Database sub-plugin for `plugin-oci` enables Kestra flows to manage Oracle Autonomous Database (ADB) instances and MySQL HeatWave clusters — starting, stopping, scaling, and monitoring database lifecycle from within a flow. This is the critical missing link for teams that want to spin up ephemeral ADB instances for data pipelines and tear them down automatically when the pipeline completes.
## Motivation
Analytics and data engineering teams using OCI Autonomous Database face a common problem: ADB costs money even when idle, so they want to stop it after batch workloads finish and restart it before the next run. Without a Kestra plugin, this requires OCI CLI wrappers or Terraform, both of which are clunky as pre/post-steps in a data pipeline. A native Database sub-plugin makes lifecycle management a first-class flow concern alongside the actual data processing tasks.
## Context
Part of the OCI Plugin Suite EPIC: https://github.com/kestra-io/plugin-oci/issues/2
Reference: `plugin-aws` RDS tasks for database lifecycle patterns.
## API Reference
- **Official docs**: https://docs.oracle.com/en-us/iaas/api/#/en/database/latest/
- **Authentication**: Config-file (`~/.oci/config`), instance principal, or `SimpleAuthenticationDetailsProvider`
- **Base URL pattern**: `https://database.{region}.oraclecloud.com/20160918/`
- **SDK**: OCI Java SDK v3.87.0 via BOM
## Gradle Dependencies
Add to `build.gradle`:
```groovy
// OCI Java SDK BOM
implementation platform("com.oracle.oci.sdk:oci-java-sdk-bom:3.87.0")
// Database service
implementation "com.oracle.oci.sdk:oci-java-sdk-database"
implementation "com.oracle.oci.sdk:oci-java-sdk-common"
```
## Plugin Structure
- **Repository**: `plugin-oci`
- **Namespace**: `io.kestra.plugin.oci.database`
- **Sub-plugins**: `database` (ADB, MySQL HeatWave)
## Suggested Tasks
1. `CreateAutonomousDatabase` — provision a new ADB instance (OLTP or DW workload type)
2. `StartAutonomousDatabase` — start a stopped ADB instance
3. `StopAutonomousDatabase` — stop a running ADB instance to pause billing
4. `DeleteAutonomousDatabase` — terminate an ADB instance
5. `GetAutonomousDatabase` — fetch ADB details and emit connection string as output
6. `ListAutonomousDatabases` — list ADB instances in a compartment
7. `ScaleAutonomousDatabase` — update OCPU count or storage size
8. Add `AutonomousDatabaseStateTrigger` — poll until ADB reaches target lifecycle state
9. Write unit + integration tests
## YAML Examples
### Example 1 — Start ADB, run a pipeline, then stop it
```yaml
id: adb_batch_pipeline
namespace: company.data
tasks:
- id: start_adb
type: io.kestra.plugin.oci.database.StartAutonomousDatabase
region: eu-frankfurt-1
tenancyOcid: "{{ secret('OCI_TENANCY_OCID') }}"
userId: "{{ secret('OCI_USER_OCID') }}"
fingerprint: "{{ secret('OCI_FINGERPRINT') }}"
privateKey: "{{ secret('OCI_PRIVATE_KEY') }}"
autonomousDatabaseId: "{{ secret('ADB_OCID') }}"
waitForLifecycleState: AVAILABLE
- id: run_etl
type: io.kestra.plugin.jdbc.oracle.Query
url: "{{ outputs.start_adb.connectionString }}"
username: "{{ secret('ADB_USER') }}"
password: "{{ secret('ADB_PASSWORD') }}"
sql: EXEC my_etl_package.run_daily;
- id: stop_adb
type: io.kestra.plugin.oci.database.StopAutonomousDatabase
region: eu-frankfurt-1
tenancyOcid: "{{ secret('OCI_TENANCY_OCID') }}"
userId: "{{ secret('OCI_USER_OCID') }}"
fingerprint: "{{ secret('OCI_FINGERPRINT') }}"
privateKey: "{{ secret('OCI_PRIVATE_KEY') }}"
autonomousDatabaseId: "{{ secret('ADB_OCID') }}"
```
### Example 2 — List all ADB instances and log their states
```yaml
id: audit_adb_instances
namespace: company.data
tasks:
- id: list_adb
type: io.kestra.plugin.oci.database.ListAutonomousDatabases
region: eu-frankfurt-1
tenancyOcid: "{{ secret('OCI_TENANCY_OCID') }}"
userId: "{{ secret('OCI_USER_OCID') }}"
fingerprint: "{{ secret('OCI_FINGERPRINT') }}"
privateKey: "{{ secret('OCI_PRIVATE_KEY') }}"
compartmentId: "{{ secret('OCI_COMPARTMENT_OCID') }}"
- id: log_states
type: io.kestra.plugin.core.log.Log
message: "Found {{ outputs.list_adb.count }} ADB instances"
```
### Example 3 — Trigger a flow when an ADB instance becomes AVAILABLE
```yaml
id: on_adb_available
namespace: company.data
triggers:
- id: adb_state_watcher
type: io.kestra.plugin.oci.database.AutonomousDatabaseStateTrigger
region: eu-frankfurt-1
tenancyOcid: "{{ secret('OCI_TENANCY_OCID') }}"
userId: "{{ secret('OCI_USER_OCID') }}"
fingerprint: "{{ secret('OCI_FINGERPRINT') }}"
privateKey: "{{ secret('OCI_PRIVATE_KEY') }}"
autonomousDatabaseId: "{{ secret('ADB_OCID') }}"
targetLifecycleState: AVAILABLE
interval: PT2M
tasks:
- id: notify
type: io.kestra.plugin.core.log.Log
message: "ADB {{ trigger.autonomousDatabaseId }} is AVAILABLE — connection: {{ trigger.connectionString }}"
```
## Acceptance Criteria
- [ ] `CreateAutonomousDatabase`, `StartAutonomousDatabase`, `StopAutonomousDatabase`, `DeleteAutonomousDatabase`, `GetAutonomousDatabase`, `ListAutonomousDatabases`, `ScaleAutonomousDatabase` tasks implemented
- [ ] `AutonomousDatabaseStateTrigger` polling trigger implemented
- [ ] `GetAutonomousDatabase` emits connection strings as typed outputs
- [ ] All `Property` fields support Kestra expression language
- [ ] Unit + integration tests pass (`./gradlew test`)
- [ ] `package-info.java` with `@PluginSubGroup(category = PluginSubGroup.PluginCategory.CLOUD)`
- [ ] Build passes with `./gradlew build`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the existing plugin-oci structure and the plugin-aws RDS tasks, then consult the OCI Database API reference and the requested dependencies in build.gradle. Implement the seven database tasks, state trigger, outputs, expression fields, package-info.java, and unit/integration tests described in the acceptance criteria. Verify with ./gradlew test and ./gradlew build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- cloud, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100