kestra-io / kestra-io/plugin-scaleway

Scaleway plugin — Serverless Functions and Containers tasks (deploy, invoke, redeploy)

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
area/plugin
Dominant language
Java
Stars
0
Forks
0
Avg merge
7h 18m
Merged PRs (30d)
2

Description

## Summary

Implement tasks for [Scaleway Serverless Functions](https://www.scaleway.com/en/serverless-functions/) and [Serverless Containers](https://www.scaleway.com/en/serverless-containers/) — Scaleway's FaaS and CaaS managed compute platforms. This enables Kestra users to deploy, invoke, and manage serverless workloads from their orchestration flows.

## Motivation

Serverless compute is a common execution layer for lightweight, event-driven processing tasks in data and platform pipelines. Teams using Scaleway Serverless today need to deploy code via the Console or CLI, then trigger it through HTTP calls from custom tasks. Native Kestra tasks will enable:

- CI/CD-style deployments: upload code → deploy function → verify invocation within a single flow
- Using serverless functions as cheap, ephemeral compute steps within a Kestra pipeline
- Managing namespaces and functions as infrastructure (create/teardown for each environment)

## Context

Sub-issue of the Scaleway plugin EPIC. Modeled after `io.kestra.plugin.aws.lambda` for function invocation patterns. Scaleway Serverless Functions API: `https://api.scaleway.com/functions/v1beta1/regions/{region}/`. Containers API: `https://api.scaleway.com/containers/v1/regions/{region}/`.

## API Reference

- **Functions docs**: https://www.scaleway.com/en/developers/api/serverless-functions/
- **Containers docs**: https://www.scaleway.com/en/developers/api/serverless-containers/
- **Authentication**: `X-Auth-Token: ` header
- **Base URL patterns**:
- Functions: `https://api.scaleway.com/functions/v1beta1/regions/{region}/`
- Containers: `https://api.scaleway.com/containers/v1/regions/{region}/`
- **SDK / client library**: No official Java SDK — use Apache HttpClient 5 + Jackson

## Gradle Dependencies

```groovy
// HTTP client for Scaleway REST API
implementation "org.apache.httpcomponents.client5:httpclient5:5.3.1"

// JSON serialization
implementation "com.fasterxml.jackson.core:jackson-databind:2.17.0"
```

> Use the latest stable versions available on Maven Central.

## Plugin Structure

- **Repository**: `plugin-scaleway`
- **Namespace**: `io.kestra.plugin.scaleway.functions` and `io.kestra.plugin.scaleway.containers`
- **Sub-plugins**: `functions`, `containers`
- **Categories**: `CLOUD`

## Suggested Tasks

### Serverless Functions
1. `ListFunctionNamespaces` — list all function namespaces in a region
2. `CreateFunctionNamespace` — create a new namespace
3. `DeleteFunctionNamespace` — delete a namespace and all its functions
4. `ListFunctions` — list functions in a namespace
5. `CreateFunction` — create a new function (runtime, code, env vars, memory/CPU)
6. `DeployFunction` — trigger deployment of a function; optionally wait for `ready` status
7. `DeleteFunction` — delete a function by ID
8. `InvokeFunction` — invoke a deployed function via its HTTP endpoint; returns response body

### Serverless Containers
1. `ListContainerNamespaces` — list container namespaces in a region
2. `CreateContainerNamespace` — create a new container namespace
3. `ListContainers` — list containers in a namespace
4. `CreateContainer` — create a new container from a registry image
5. `RedeployContainer` — redeploy a container (new image pull + restart); wait for `ready`
6. `DeleteContainer` — delete a container by ID

## YAML Examples

### Example 1 — Deploy a new Scaleway Function from a ZIP archive

```yaml
id: deploy_scaleway_function
namespace: company.platform

inputs:
- id: function_name
type: STRING

tasks:
- id: create_namespace
type: io.kestra.plugin.scaleway.functions.CreateFunctionNamespace
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
name: "{{ inputs.function_name }}-ns"
runtime: python311

- id: create_function
type: io.kestra.plugin.scaleway.functions.CreateFunction
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
namespaceId: "{{ outputs.create_namespace.namespaceId }}"
name: "{{ inputs.function_name }}"
runtime: python311
handler: handler.handle
memoryLimit: 256

- id: deploy
type: io.kestra.plugin.scaleway.functions.DeployFunction
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
functionId: "{{ outputs.create_function.functionId }}"
waitForReady: true

- id: invoke
type: io.kestra.plugin.scaleway.functions.InvokeFunction
endpoint: "{{ outputs.create_function.domainName }}"
payload: '{"key": "value"}'
```

### Example 2 — Redeploy a container when a new Docker image is pushed

```yaml
id: redeploy_on_image_push
namespace: company.platform

tasks:
- id: redeploy_api
type: io.kestra.plugin.scaleway.containers.RedeployContainer
secretKey: "{{ secret('SCW_SECRET_KEY') }}"
region: fr-par
containerId: "{{ secret('SCW_API_CONTAINER_ID') }}"
waitForReady: true

- id: verify
type: io.kestra.plugin.core.log.Log
message: "Container redeployed successfully. Status: {{ outputs.redeploy_api.status }}"
```

### Example 3 — Invoke a function as a processing step and use its output

```yaml
id: process_with_serverless
namespace: company.data

tasks:
- id: invoke_transform
type: io.kestra.plugin.scaleway.functions.InvokeFunction
endpoint: "{{ secret('SCW_TRANSFORM_FUNCTION_URL') }}"
payload: |
{
"dataset": "{{ inputs.dataset_uri }}",
"format": "parquet"
}

- id: log_result
type: io.kestra.plugin.core.log.Log
message: "Function returned: {{ outputs.invoke_transform.responseBody }}"
```

## Acceptance Criteria

- [ ] `ListFunctionNamespaces`, `CreateFunctionNamespace`, `DeleteFunctionNamespace` implemented
- [ ] `ListFunctions`, `CreateFunction`, `DeployFunction`, `DeleteFunction`, `InvokeFunction` implemented
- [ ] `ListContainerNamespaces`, `CreateContainerNamespace` implemented
- [ ] `ListContainers`, `CreateContainer`, `RedeployContainer`, `DeleteContainer` implemented
- [ ] `DeployFunction` and `RedeployContainer` support `waitForReady` polling flag
- [ ] `InvokeFunction` supports payload and returns response body as task output
- [ ] All `Property` fields support Kestra expression language (template rendering)
- [ ] Unit tests with mocked HTTP responses; integration tests against live Scaleway (skipped if credentials absent)
- [ ] `package-info.java` per sub-package with `@PluginSubGroup(category = PluginSubGroup.PluginCategory.CLOUD)`
- [ ] Build passes with `./gradlew build`

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the existing io.kestra.plugin.aws.lambda implementation and the Scaleway Functions and Containers API references. Organize the work under the proposed functions and containers namespaces, including package-info.java, then add mocked HTTP tests and optional live integration tests. Done means all listed tasks, polling, templating, outputs, and ./gradlew build satisfy the acceptance criteria.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.