kestra-io / kestra-io/plugin-scaleway

Scaleway plugin — Compute Instances tasks (provision, start/stop, terminate)

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
area/plugin
Dominant language
Java
Stars
0
Forks
0
Avg merge
7h 18m
Merged PRs (30d)
2

Description

## Summary

Implement tasks for [Scaleway Instances](https://www.scaleway.com/en/virtual-instances/) (compute) — Scaleway's virtual machine service. This enables Kestra users to provision, manage, and deprovision Scaleway VMs as part of infrastructure automation, batch compute, and CI/CD workflows.

## Motivation

Platform and infrastructure teams often need to spin up ephemeral compute resources for heavy workloads (model training, large-scale data processing, integration testing environments), run them for the duration of a pipeline, then terminate them. Today, Scaleway users must use shell scripts or custom HTTP tasks to interact with the Instances API. Native tasks enable:

- Infrastructure-as-code workflows: create instances on demand, run workloads, then terminate
- Automated environment provisioning for staging/QA pipelines
- Cost-efficient batch compute: only pay for runtime while the Kestra flow runs

## Context

Sub-issue of the Scaleway plugin EPIC. Reference implementation: `io.kestra.plugin.gcp.compute` in `plugin-gcp`. Scaleway Instances API is a zonal API: `https://api.scaleway.com/instance/v1/zones/{zone}/`.

## API Reference

- **Official docs**: https://www.scaleway.com/en/developers/api/instances/
- **Authentication**: `X-Auth-Token: ` header
- **Base URL pattern**: `https://api.scaleway.com/instance/v1/zones/{zone}/`
- **Available zones**: `fr-par-1`, `fr-par-2`, `fr-par-3`, `nl-ams-1`, `nl-ams-2`, `nl-ams-3`, `pl-waw-1`, `pl-waw-2`, `pl-waw-3`
- **SDK / client library**: No official Java SDK — use Apache HttpClient 5 + Jackson

## Gradle Dependencies

```groovy
// HTTP client for Scaleway REST API
implementation "org.apache.httpcomponents.client5:httpclient5:5.3.1"

// JSON serialization
implementation "com.fasterxml.jackson.core:jackson-databind:2.17.0"
```

> Use the latest stable versions available on Maven Central.

## Plugin Structure

- **Repository**: `plugin-scaleway`
- **Namespace**: `io.kestra.plugin.scaleway.instance`
- **Sub-plugins**: `instance`
- **Categories**: `CLOUD`

## Suggested Tasks

1. `AbstractInstance` — abstract base class with `secretKey`, `zone` as `Property`; builds an authenticated HTTP client
2. `ListServers` — list all servers in a zone with optional filtering by state/name; outputs list of server summaries
3. `GetServer` — get details for a specific server by ID; outputs server state, IP, type
4. `CreateServer` — create a new server with specified type, image, and name; waits for RUNNING state
5. `DeleteServer` — delete a server by ID; optionally wait for deletion to complete
6. `ServerAction` — perform an action on a server (start / stop / reboot / poweroff / terminate)
7. `ListServerTypes` — list available commercial server types (DEV1, GP1, etc.) with specs

## YAML Examples

### Example 1 — Provision an ephemeral instance, run a script, then terminate

```yaml
id: ephemeral_batch_compute
namespace: company.platform

tasks:
- id: create_instance
type: io.kestra.plugin.scaleway.instance.CreateServer
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
zone: fr-par-1
name: "kestra-batch-{{ execution.id }}"
commercialType: DEV1-S
image: ubuntu_jammy
waitForReady: true

- id: log_ip
type: io.kestra.plugin.core.log.Log
message: "Instance {{ outputs.create_instance.serverId }} running at {{ outputs.create_instance.publicIp }}"

- id: terminate_instance
type: io.kestra.plugin.scaleway.instance.ServerAction
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
zone: fr-par-1
serverId: "{{ outputs.create_instance.serverId }}"
action: terminate
```

### Example 2 — List all running instances and log their names

```yaml
id: audit_scaleway_instances
namespace: company.platform

tasks:
- id: list_running
type: io.kestra.plugin.scaleway.instance.ListServers
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
zone: fr-par-1
state: running

- id: log_count
type: io.kestra.plugin.core.log.Log
message: "{{ outputs.list_running.total }} instances running in fr-par-1"
```

### Example 3 — Scheduled nightly shutdown of dev instances

```yaml
id: nightly_instance_shutdown
namespace: company.platform

triggers:
- id: nightly
type: io.kestra.plugin.core.trigger.Schedule
cron: "0 22 * * 1-5"

tasks:
- id: list_dev_instances
type: io.kestra.plugin.scaleway.instance.ListServers
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
zone: fr-par-1
tags:
- env:dev

- id: stop_each
type: io.kestra.plugin.core.flow.ForEach
values: "{{ outputs.list_dev_instances.servers }}"
tasks:
- id: stop
type: io.kestra.plugin.scaleway.instance.ServerAction
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
zone: fr-par-1
serverId: "{{ taskrun.value.id }}"
action: poweroff
```

## Acceptance Criteria

- [ ] `AbstractInstance` base class with `secretKey`, `zone` as `Property`
- [ ] `ListServers`, `GetServer`, `CreateServer`, `DeleteServer`, `ServerAction` tasks implemented
- [ ] `CreateServer` supports `waitForReady` flag polling until state is `running`
- [ ] `ServerAction` supports: `start`, `stop`, `reboot`, `poweroff`, `terminate`
- [ ] All `Property` fields support Kestra expression language (template rendering)
- [ ] Unit tests with mocked HTTP responses; integration tests against live Scaleway (skipped if credentials absent)
- [ ] `package-info.java` with `@PluginSubGroup(category = PluginSubGroup.PluginCategory.CLOUD)`
- [ ] Build passes with `./gradlew build`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the reference implementation in plugin-gcp and the Scaleway Instances API documentation, then map the requested tasks under the io.kestra.plugin.scaleway.instance namespace. Check package-info.java and the Gradle dependency requirements, add mocked HTTP and optional live integration tests, and run ./gradlew build to verify completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.