argoproj / argoproj/argo-workflows

Multicluster: adopting a standardized ClusterInventory API

Open
#11,389 1 comment 14 reactions 0 assignees View on GitHub
area/multi-cluster
Dominant language
Go
Stars
17k
Forks
3.7k
Avg merge
1d 20h
Merged PRs (30d)
138

Description

Opening this issue here but TBD on whether the implementation should occur directly in this repository, or if it should be kept in a separate repository and outside of the Workflow Controller in the interest of not creating new bugs here.

**************************************************************************************************************

Argo Workflows can benefit from adopting a standardized [ClusterInventory API](https://docs.google.com/document/d/1sUWbe81BTclQ4Uax3flnCoKtEWngH-JA9MyCqljJCBM/) and expanding its multi-cluster capabilities. By doing so, it can seamlessly support various multi-cluster projects without the need to import different project APIs.
By integrating with a Cluster Inventory API, Argo Workflows can immediately retrieve the status information for a specific cluster. This information can then be used as a templating value within a Workflow, allowing the Workflow to utilize the specific details of that cluster during its execution.
For example:

```yaml
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: hello-world-parameters-
spec:
entrypoint: whalesay
templates:
- name: whalesay
inputs:
clusterInventories: # list of cluster inventories
- cluster1
container:
image: docker/whalesay
command: [cowsay]
args: ["{{inputs.clusterInventories.cluster1.name}}", "{{inputs.clusterInventories.cluster1.allocatableMemory}}"] # cluster inventory templating
```

In the future, Argo Workflows has the potential to leverage the Cluster Inventory to determine the availability and resource capacities of registered clusters. This valuable information can significantly enhance workflow scheduling, ensuring optimal resource utilization across clusters. Argo Workflow would be able to intelligently distribute tasks or stages of a workflow to different clusters based on their capacity and current workload, further optimizing the overall performance

---

**Message from the maintainers**:

Love this enhancement proposal? Give it a 👍. We prioritise the proposals with the most 👍.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the linked ClusterInventory API proposal and resolving whether the implementation belongs in this repository or a separate one. Then define how cluster status would be retrieved and exposed as workflow templating values, using the YAML example as the acceptance target. Future scheduling and resource-capacity behavior is described as a later possibility.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes
Domain
api, distributed-systems, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.