kubevela / kubevela/workflow

Standalone vela-workflow ignores kube.#Apply cluster parameter and applies resources to the control-plane cluster

Open
#250 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
126
Forks
66
Avg merge
1h 40m
Merged PRs (30d)
1

Description

**Describe the bug**

When running the standalone `vela-workflow` controller, the `cluster` parameter of the `vela/kube` provider is ignored in practice.

A custom WorkflowStepDefinition using `kube.#Apply` with a managed cluster name attempts to create the resource in the control-plane cluster instead. If the target namespace exists only in the managed cluster, the step fails with:

```text
namespaces "remote-workflow-test" not found
```

The rendered action contains the correct cluster:

```cue
$params: {
cluster: "managed-cluster"
value: {
...
metadata: {
namespace: "remote-workflow-test"
}
}
}
```

However, the request is sent to the control-plane cluster.

**To Reproduce**

1. Register a managed cluster named `managed-cluster`.

2. Create the namespace only in the managed cluster:

```bash
kubectl --context managed-cluster create namespace remote-workflow-test
```

Do not create this namespace in the control-plane cluster.

3. Apply the following WorkflowStepDefinition:

```yaml
apiVersion: core.oam.dev/v1beta1
kind: WorkflowStepDefinition
metadata:
name: apply-to-managed-cluster
namespace: vela-system
spec:
schematic:
cue:
template: |
import "vela/kube"

apply: kube.#Apply & {
$params: {
cluster: parameter.cluster
value: {
apiVersion: "v1"
kind: "ConfigMap"
metadata: {
name: "multi-cluster-test"
namespace: parameter.namespace
}
data: {
result: "created"
}
}
}
}

parameter: {
cluster: string
namespace: string
}
```

4. Apply the following WorkflowRun:

```yaml
apiVersion: core.oam.dev/v1alpha1
kind: WorkflowRun
metadata:
name: apply-to-managed-cluster-test
namespace: default
spec:
workflowSpec:
steps:
- name: apply
type: apply-to-managed-cluster
properties:
cluster: managed-cluster
namespace: remote-workflow-test
```

5. Check the WorkflowRun:

```bash
kubectl -n default get workflowrun apply-to-managed-cluster-test -o yaml
```

The step fails with:

```text
function call error for apply: namespaces "remote-workflow-test" not found
```

6. Verify that the namespace exists and is accessible through ClusterGateway:

```bash
kubectl --cluster managed-cluster get namespace remote-workflow-test
```

This succeeds, while the same query against the control-plane cluster returns `NotFound`.

**Expected behavior**

`kube.#Apply` should use the multi-cluster client and create the ConfigMap in:

```text
managed-cluster/remote-workflow-test
```

The WorkflowRun should succeed, and the following command should return the ConfigMap:

```bash
kubectl --cluster managed-cluster \
-n remote-workflow-test \
get configmap multi-cluster-test
```

No resource should be created in the control-plane cluster.

**Screenshots**

Not applicable.

**Workflow Version**

```text
vela-workflow: v0.7.2
github.com/kubevela/pkg: v1.11.1
Deployment mode: standalone vela-workflow controller
```

The controller binary exposes the following flags, but they are not related to the failure because an empty URL intentionally uses the Kubernetes Aggregated API:

```text
--cluster-gateway-url
--cluster-gateway-ca-file
```

**Cluster information**

```text
Control-plane cluster:
Managed cluster:
ClusterGateway: enabled
ClusterGateway APIService: Available
```

The managed cluster can be accessed successfully through the exported ClusterGateway kubeconfig.

**Additional context**

The standalone controller creates its Manager with the correct multi-cluster client:

```go
mgr, err := ctrl.NewManager(restConfig, ctrl.Options{
NewClient: velaclient.DefaultNewControllerClient,
})
```

`DefaultNewControllerClient` internally calls:

```go
multicluster.NewDefaultClient(...)
```

However, `WorkflowRunReconciler.Reconcile` does not inject `r.Client` into the provider context.

When the kube provider resolves its runtime client, this branch is not taken:

```go
if kubeClient, ok := ctx.Value(KubeClientKey).(client.Client); ok {
params.KubeClient = kubeClient
} else {
params.KubeClient = singleton.KubeClient.Get()
}
```

It falls back to `singleton.KubeClient`, which is created using the regular controller-runtime client:

```go
var KubeClient = NewSingletonE[client.Client](func() (client.Client, error) {
return client.New(KubeConfig.Get(), client.Options{
Scheme: scheme.Scheme,
Mapper: RESTMapper.Get(),
})
})
```

This client does not consume `DefaultClusterGatewayClientOptions` and does not interpret the cluster value added by:

```go
multicluster.WithCluster(ctx, cluster)
```

A possible fix is to inject the Reconciler's multi-cluster client into the context:

```go
func WithKubeClient(parent context.Context, kubeClient client.Client) context.Context {
return context.WithValue(parent, KubeClientKey, kubeClient)
}
```

Then use it at the beginning of `WorkflowRunReconciler.Reconcile`:

```go
ctx = types.SetNamespaceInCtx(ctx, req.Namespace)
ctx = providertypes.WithKubeClient(ctx, r.Client)
```

After applying this change, the same WorkflowRun successfully creates the resource in the specified managed cluster.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at WorkflowRunReconciler.Reconcile and trace how its client reaches the kube provider's runtime client resolution, including DefaultNewControllerClient and the KubeClientKey fallback. Reproduce the WorkflowRun against the managed and control-plane clusters, then verify that the run succeeds and the ConfigMap appears only in managed-cluster/remote-workflow-test.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, distributed-systems
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
62/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.