pipe-cd / pipe-cd/pipecd

Make PipeCD control plane support plugin-arched piped

Open
#5,252 24 comments 0 reactions 3 assignees View on GitHub

@hongky-1994 is already working on this.

Since Jan 30, 2025.

area/pipedv1 epic not-auto-close Stale
Dominant language
Go
Stars
1.4k
Forks
364
Avg merge
1d 4h
Merged PRs (30d)
84

Description

What would you like to be added:

This issue tracks the progress for deprecating Platform Provider and Kind toward piped plugin architecture in pipedv1.

The purpose is to manage application kind and platform providers on the PipeCD plugin side in the future.

Considering the impact on users, pay attention to the following points:

  • In Pipedv1, platform providers, kind remain on the data and do not create logic with them. Use the value you define instead.
  • minify the migration for users.

current update process

  1. Update data for the pipedv1
  • Add the label kind based on the current Application Kind to the Application.
  • Add deployTarget based on the current Application Platform Provider to the Application.
  1. update to pipedv1 (users can use existing application at the time)

Details (under planning)

For Platform Provider

Instead of Platform Provider, we plan to introduce the config for the plugin and define deployTargets.

piped config

apiVersion: pipecd.dev/v1beta1
kind: Piped
spec:
...
  plugin:
    - name: k8s_plugin
      port: 8081
      deployTargets:
        - name: dev
          labels: 
            env: dev
          config: # depends on plugins
            masterURL: http://cluster-dev
            kubeConfigPath: ./kubeconfig-dev

type PipedDeployTarget struct {
	Name   string                     `json:"name"`
	Labels map[string]string          `json:"labels,omitempty"`
	Config json.RawMessage            `json:"config"`
}

We also plan to deploy the app to multiple targets at once in a multicluster feature for k8s.
So, we define DeployTargets as an array in Application and Deployment.

Application

message Application {
    reserved 3;
    ...
    // TODO: Add validation for this field.
    string platform_provider = 15;
    // 
    repeated string deploy_targets = 16;
    ...
}

Deployment

message Deployment {
    reserved 4;
    ...
    // The name of platform provider where to deploy this application.
    // This must be one of the provider names registered in the piped.
    string platform_provider = 11;
    
    repeated string deploy_targets = 12;
}
For the backward compatibility

Before updating the piped, users should migrate the current data on DB.

Former idea

During the migration, there might be both platform providers and deploy targets in the piped config.
So we need to convert the platform providers to deploy targets internally.

Refer the Platform Provider or Deploy Target

  • If the ApplicationKind is Application, just use DeployTarget
  • If the ApplicationKind is old one, convert PlatformProvider to DeployTarget
func (s *PipedSpec) FindDeployTarget(name string, t model.ApplicationKind) (*PipedDeployTarget, bool) {
	// First, check the application is supported by the plugin architecture. It means that the kind is set to "Application".
	// If not, the deploy target is the platform provider.
	// For backward compatibility, the deploy target is the platform provider.
	if t != model.ApplicationKind_APPLICATION {
		p, found := s.FindPlatformProvider(name, t)
		if !found {
			return &PipedDeployTarget{}, false
		}
		return &PipedDeployTarget{
				Name:   p.Name,
				Labels: p.Labels,
				Config: p.Config,
			}, true
	}

	// If the application is supported by the plugin architecture, the deploy target is the deploy target.
	for _, dt := range s.DeployTargets {
		if dt.Name == name {
			return dt, true
		}
	}

	return &PipedDeployTarget{}, false
}
For Kind

Instead of Kind, we plan to introduce the label to represent the application kind.

apiVersion: pipecd.dev/v1beta1
kind: Application
metadata:
  labels:
    kind: KUBERNETES # <- like this
spec:
  name: myApp

For the builtin plugins, we define 5 labels as string.

  • KUBERNETES
  • ECS
  • LAMBDA
  • CLOUDRUN
  • TERRAFORM
For the backward compatibility

Before updating the piped, users should migrate the current data on DB.

Former idea

We need to support both before and after creating plugin architecture for now.
So, I propose the way to decide the application kind like this.

  • Define APPLICATION as ApplicationKind
  • Add new method to decide the actual kind for Application and Deployment.
enum ApplicationKind {
    KUBERNETES = 0;
    TERRAFORM = 1;
    LAMBDA = 3;
    CLOUDRUN = 4;
    ECS = 5;
    APPLICATION = 6; <- new! 
}

func (a * Application) GetKind() string {
	// First, check the application is supported by the plugin architecture. It means that the kind is set to "Application".
	// If so, return the kind from the labels.
	if a.Kind == ApplicationKind_Application {
		return a.Labels["kind"]
	}

	// For backward compatibility, return the kind as string
	return a.Kind.String()
}

func (d *Deployment) GetKind() string {
	// First, check the application is supported by the plugin architecture. It means that the kind is set to "Application".
	// If so, return the kind from the labels.
	if d.Kind == ApplicationKind_Application {
		return d.Labels["kind"]
	}

	// For backward compatibility, return the kind as string
	return d.Kind.String()
}

TODO

Firstly, I listed the target features to require some modifications.
We need to further investigation to decide the way to fix.

investigation

implementation

Why is this needed:

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.