kestra-io / kestra-io/plugin-scaleway
Scaleway plugin — Managed Databases (RDB) tasks (provision, backup, restore)
- Dominant language
- Java
- Stars
- 0
- Forks
- 0
- Avg merge
- 7h 18m
- Merged PRs (30d)
- 2
Description
## Summary
Implement tasks for [Scaleway Managed Databases (RDB)](https://www.scaleway.com/en/database/) — Scaleway's managed PostgreSQL and MySQL service. This enables Kestra users to provision, manage, and automate backup/restore of managed database instances as part of data platform and infrastructure workflows.
## Motivation
Managed databases are a critical component of data platform architectures. Teams managing Scaleway RDB instances today need the Scaleway Console or CLI to create/scale/delete instances, manage databases and users, and trigger backups. Native Kestra tasks enable:
- Database provisioning as part of environment setup flows (staging, feature branch deployments)
- Automated backup scheduling and point-in-time restore verification
- On-demand instance scaling before/after heavy ETL workloads
- Cleanup of ephemeral database instances after integration tests
## Context
Sub-issue of the Scaleway plugin EPIC. Scaleway Managed Database API: `https://api.scaleway.com/rdb/v1/regions/{region}/`. Supports PostgreSQL (14, 15, 16) and MySQL (8). Note: this issue covers *infrastructure management* of RDB instances; actual SQL querying should use Kestra's existing `plugin-jdbc-postgresql` and `plugin-jdbc-mysql` plugins.
## API Reference
- **Official docs**: https://www.scaleway.com/en/developers/api/managed-databases-for-postgresql-and-mysql/
- **Authentication**: `X-Auth-Token: ` header
- **Base URL pattern**: `https://api.scaleway.com/rdb/v1/regions/{region}/`
- **Available regions**: `fr-par`, `nl-ams`, `pl-waw`
- **SDK / client library**: No official Java SDK — use Apache HttpClient 5 + Jackson
## Gradle Dependencies
```groovy
// HTTP client for Scaleway REST API
implementation "org.apache.httpcomponents.client5:httpclient5:5.3.1"
// JSON serialization
implementation "com.fasterxml.jackson.core:jackson-databind:2.17.0"
```
> Use the latest stable versions available on Maven Central.
## Plugin Structure
- **Repository**: `plugin-scaleway`
- **Namespace**: `io.kestra.plugin.scaleway.rdb`
- **Sub-plugins**: `rdb`
- **Categories**: `CLOUD`
## Suggested Tasks
1. `AbstractRdb` — abstract base class with `secretKey`, `region` as `Property`
2. `ListInstances` — list all RDB instances in a region with optional filtering by engine/status
3. `GetInstance` — get details for a specific instance (endpoint, status, node type)
4. `CreateInstance` — create a new managed database instance; wait for `ready` status
5. `DeleteInstance` — delete an instance (with data loss warning in docs)
6. `UpgradeInstance` — upgrade node type or storage volume of an existing instance
7. `CreateDatabase` — create a logical database within an instance
8. `ListDatabases` — list databases within an instance
9. `CreateUser` — create a database user with grants
10. `CreateBackup` — trigger an on-demand backup of an instance
11. `ListBackups` — list available backups for an instance
12. `RestoreBackup` — restore a backup to a new instance
## YAML Examples
### Example 1 — Provision a staging database instance for a feature branch
```yaml
id: provision_staging_db
namespace: company.platform
inputs:
- id: branch_name
type: STRING
tasks:
- id: create_instance
type: io.kestra.plugin.scaleway.rdb.CreateInstance
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
name: "{{ inputs.branch_name }}-db"
engine: PostgreSQL-16
nodeType: DB-DEV-S
isHaCluster: false
userName: appuser
password: "{{ secret('STAGING_DB_PASSWORD') }}"
waitForReady: true
- id: create_database
type: io.kestra.plugin.scaleway.rdb.CreateDatabase
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
instanceId: "{{ outputs.create_instance.instanceId }}"
name: appdb
- id: log_endpoint
type: io.kestra.plugin.core.log.Log
message: "DB ready at {{ outputs.create_instance.endpoint.ip }}:{{ outputs.create_instance.endpoint.port }}"
```
### Example 2 — Trigger a backup before a major schema migration
```yaml
id: pre_migration_backup
namespace: company.data
tasks:
- id: backup
type: io.kestra.plugin.scaleway.rdb.CreateBackup
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
instanceId: "{{ secret('PROD_DB_INSTANCE_ID') }}"
name: "pre-migration-{{ execution.startDate | dateFormat('yyyyMMdd-HHmm') }}"
- id: confirm
type: io.kestra.plugin.core.log.Log
message: "Backup {{ outputs.backup.backupId }} created. Proceeding with migration."
```
### Example 3 — List backups and log the most recent one
```yaml
id: audit_backups
namespace: company.platform
tasks:
- id: list_backups
type: io.kestra.plugin.scaleway.rdb.ListBackups
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
instanceId: "{{ secret('PROD_DB_INSTANCE_ID') }}"
- id: log_latest
type: io.kestra.plugin.core.log.Log
message: "Latest backup: {{ outputs.list_backups.backups | first | jq('.name') }} created at {{ outputs.list_backups.backups | first | jq('.createdAt') }}"
```
## Acceptance Criteria
- [ ] `AbstractRdb` base class with `secretKey`, `region` as `Property`
- [ ] `ListInstances`, `GetInstance`, `CreateInstance`, `DeleteInstance`, `UpgradeInstance` implemented
- [ ] `CreateInstance` supports `waitForReady` flag polling until status is `ready`
- [ ] `CreateDatabase`, `ListDatabases`, `CreateUser` implemented
- [ ] `CreateBackup`, `ListBackups`, `RestoreBackup` implemented
- [ ] All `Property` fields support Kestra expression language (template rendering)
- [ ] Unit tests with mocked HTTP responses; integration tests against live Scaleway (skipped if credentials absent)
- [ ] `package-info.java` with `@PluginSubGroup(category = PluginSubGroup.PluginCategory.CLOUD)`
- [ ] Build passes with `./gradlew build`
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the suggested plugin structure and the Scaleway Managed Databases API reference, then inspect the existing repository layout for analogous plugins. Implement the listed RDB tasks with mocked HTTP tests, add package-info.java with the CLOUD category, and verify completion with ./gradlew build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, mysql, postgresql
- Domain
- cloud, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100