apache / apache/ignite

Enable Kubernetes Parallel pod management policy

Open
#11,022 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
5.1k
Forks
1.9k
Avg merge
3d 2h
Merged PRs (30d)
46

Description

## Problem:
Currently, the Kubernetes IP Finder requires configuring a [Kubernetes Service](https://kubernetes.io/docs/concepts/services-networking/service/) name, and [the module queries](https://github.com/apache/ignite/blob/27ed13a607815b385085b3d629e961265ff64b54/modules/kubernetes/src/main/java/org/apache/ignite/internal/kubernetes/connection/KubernetesServiceAddressResolver.java#L95) [this object in the Kubernetes API](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#service-v1-core) to enumerate Ignite cluster node IP addresses. Kubernetes will list only those nodes which have already successfully become ready to receive query traffic. OrderedReady pod management policy in StatefulSet is the default, and works well, but enforces the slowest cluster startup as each node must present itself as ready prior to the next node's pod startup. Parallel pod management policy does not work because all nodes perform and complete the discovery phase before any nodes have presented themselves as ready for the Kubernetes Service.

## Consequences:
Large clusters (many nodes) and clusters with many cache groups may take tens of minutes (or more) to start up. Kubernetes hosted Ignite clusters should behave more like non-Kubernetes hosted clusters with Parallel pod management policy, enabling independent nodes to perform most startup work concurrently. This proposal would enable arbitrarily large Kubernetes clusters to start up in similar time to small clusters.

## Proposal:
1. Continue to use nodes' IPs enumerated by the Service ClusterIPs or Endpoints
2. Inspect the Service spec.Selector
3. List pods constrained by the Service Selector
4. Iterate through listed pods, and query each Pod's status to collect PodIPs
5. Merge collected PodIPs to the list of IPs from the Service

## References:
* Code StatusQuo https://github.com/apache/ignite/blob/27ed13a607815b385085b3d629e961265ff64b54/modules/kubernetes/src/main/java/org/apache/ignite/internal/kubernetes/connection/KubernetesServiceAddressResolver.java#L95
* [Kubernetes StatefulSet .spec.PodManagementPolicy](https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#pod-management-policies)
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#service-v1-core
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#list-endpoints-v1-core
* https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#status-operations-pod-v1-core

Contributor guide

Open the contributing guide

Research direction

Start with modules/kubernetes/src/main/java/org/apache/ignite/internal/kubernetes/connection/KubernetesServiceAddressResolver.java and its current Service or Endpoints lookup. Read the Kubernetes Service selector, pod-listing, and Pod status references in the issue, then trace how discovered addresses are merged. Done means Service addresses remain supported and pod IPs are available during Parallel pod management startup.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kubernetes
Domain
cloud, distributed-systems
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.