Proposal: Adopt kube-authkit as Default Authentication Mechanism

## Summary

This proposal suggests adopting [kube-authkit](https://github.com/opendatahub-io/kube-authkit) as the default and unified authentication mechanism for Kubeflow SDK. This would provide a consistent, secure, and maintainable authentication layer across all Kubeflow components while reducing code duplication and improving the developer experience.

## Background

Currently, Kubeflow SDK components implement various authentication methods:
- **Pipelines SDK**: ServiceAccount tokens, OIDC browser flows, port-forwarding with kubectl
- **Training SDK**: Custom authentication implementations
- **Katib SDK**: Component-specific credential handling
- **Model Registry**: Individual authentication logic

This fragmentation leads to:
- **Code duplication** across components
- **Inconsistent authentication behavior** between Kubeflow services
- **Maintenance burden** for multiple authentication implementations
- **Security vulnerabilities** in outdated Kubernetes client dependencies (see CVE details below)
- **Poor developer experience** with different authentication patterns per component

## What is kube-authkit?

[kube-authkit](https://github.com/opendatahub-io/kube-authkit) is a unified Kubernetes authentication toolkit designed specifically for this use case. It provides a single, well-tested abstraction layer that supports:

### Supported Authentication Methods

1. **Auto-detection** - Automatically selects the best method for the environment
2. **In-Cluster** - Service account tokens (for pods)
3. **KubeConfig** - Standard `~/.kube/config` files (for local development)
4. **OIDC** - OpenID Connect with Authorization Code Flow and Device Flow
5. **OpenShift OAuth** - Native OpenShift authentication

### Key Features

- **Zero-configuration authentication** via environment variables
- **Automatic strategy selection** based on runtime environment
- **Kubernetes Secrets integration** for seamless pod authentication
- **Persistent token storage** via system keyring (optional)
- **Secure defaults** with configurable SSL/TLS verification
- **Comprehensive error messages** for troubleshooting
- **Type-safe configuration** with validation
- **Extensive test coverage** and production-ready

## Benefits for Kubeflow SDK

### 1. Unified Authentication Layer

Replace multiple authentication implementations with a single, consistent API:

**Before (Pipelines SDK):**
```python
# Different authentication patterns for different scenarios
client = kfp.Client(
host='...',
existing_token='...', # Manual token management
# or use browser flow
# or use port-forwarding
# or hope in-cluster detection works
)
```

**After (with kube-authkit):**
```python
from kube_authkit import get_k8s_client

# Auto-detects environment and authenticates
api_client = get_k8s_client()

# Or with explicit configuration
from kube_authkit import AuthConfig
config = AuthConfig(method="oidc", oidc_issuer="...", client_id="...")
api_client = get_k8s_client(config)
```

### 2. Environment Variable Configuration

Enable zero-code authentication via Kubernetes secrets:

**Kubernetes Deployment:**
```yaml
apiVersion: v1
kind: Secret
metadata:
name: kubeflow-auth
stringData:
AUTHKIT_OIDC_ISSUER: "https://your-idp.example.com"
AUTHKIT_CLIENT_ID: "kubeflow-client"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: kubeflow-pipeline
spec:
template:
spec:
containers:
- name: pipeline-runner
image: kubeflow/pipeline-runner:latest
envFrom:
- secretRef:
name: kubeflow-auth
```

**Python Code (no changes needed):**
```python
# Automatically picks up configuration from environment
api_client = get_k8s_client()
```

### 3. Security Improvements

#### CVE Mitigation

The kube-authkit project has **already upgraded to kubernetes>=35.0.0**, which fixes critical CVEs in older versions:

- **CVE-2025-66418** (HIGH) - urllib3 vulnerability in kubernetes 34.1.0 and earlier
- **CVE-2025-66471** (HIGH) - urllib3 vulnerability in kubernetes 34.1.0 and earlier
- **CVE-2026-1703** - pip vulnerability (addressed in CI/CD pipeline)

Many Kubeflow SDK components likely still depend on older `kubernetes` Python client versions with these vulnerabilities. Adopting kube-authkit provides an immediate security upgrade path.

#### Secure Defaults

- TLS/SSL verification enabled by default
- Security warnings for insecure configurations
- Sensitive credentials redacted from logs and error messages
- Support for custom CA certificates

### 4. Better Developer Experience

#### Local Development

Works seamlessly with local `kubectl` configuration:

```python
# Just works if kubectl is configured
from kube_authkit import get_k8s_client
api_client = get_k8s_client()
```

#### In-Cluster Deployment

Automatically detects in-cluster environment:

```python
# Same code works in-cluster with no changes
from kube_authkit import get_k8s_client
api_client = get_k8s_client()
```

#### OIDC Workflows

Supports both interactive and non-interactive OIDC flows:

```python
# Interactive browser flow for local development
config = AuthConfig(
method="oidc",
oidc_issuer="https://keycloak.example.com/realms/kubeflow",
client_id="kubeflow-client",
use_device_flow=False # Opens browser
)

# Device flow for headless environments
config = AuthConfig(
method="oidc",
oidc_issuer="https://keycloak.example.com/realms/kubeflow",
client_id="kubeflow-client",
use_device_flow=True # Shows device code to paste
)
```

### 5. Reduced Maintenance Burden

- **Single dependency** to update instead of multiple authentication implementations
- **Community-maintained** authentication logic
- **Comprehensive tests** for each authentication strategy
- **Well-documented** API and configuration options
- **Active development** with regular security updates

## Migration Path

### Phase 1: Opt-In Support (Backward Compatible)

Add kube-authkit as an optional dependency and provide adapter functions:

```python
# kubeflow/sdk/utils/auth.py

try:
from kube_authkit import get_k8s_client, AuthConfig
KUBE_AUTHKIT_AVAILABLE = True
except ImportError:
KUBE_AUTHKIT_AVAILABLE = False

def get_authenticated_client(
host: Optional[str] = None,
existing_token: Optional[str] = None,
use_kube_authkit: bool = True,
**legacy_kwargs
):
"""Get authenticated Kubernetes client with optional kube-authkit support."""

if use_kube_authkit and KUBE_AUTHKIT_AVAILABLE:
# Use kube-authkit if available
config = AuthConfig(
k8s_api_host=host,
token=existing_token,
)
return get_k8s_client(config)
else:
# Fall back to legacy authentication
return _legacy_authentication(host, existing_token, **legacy_kwargs)
```

### Phase 2: Default to kube-authkit

Make kube-authkit the default while maintaining backward compatibility:

```python
# pyproject.toml
dependencies = [
"kubernetes>=35.0.0",
"kube-authkit>=0.3.0",
...
]
```

### Phase 3: Deprecate Legacy Authentication

Announce deprecation timeline and provide migration guides:

```python
def get_authenticated_client(
use_kube_authkit: bool = True, # Default to True
**kwargs
):
if not use_kube_authkit:
warnings.warn(
"Legacy authentication is deprecated and will be removed in v2.0. "
"Please migrate to kube-authkit.",
DeprecationWarning,
stacklevel=2
)
```

### Phase 4: Remove Legacy Code

Clean up deprecated authentication code in a major version bump.

## Example Integration

### Kubeflow Pipelines Client

**Current Implementation:**
```python
class Client:
def __init__(
self,
host: Optional[str] = None,
client_id: Optional[str] = None,
namespace: str = "kubeflow",
other_client_id: Optional[str] = None,
other_client_secret: Optional[str] = None,
existing_token: Optional[str] = None,
cookies: Optional[str] = None,
proxy: Optional[str] = None,
ssl_ca_cert: Optional[str] = None,
credentials: Optional[client.TokenCredentialsBase] = None,
):
# Complex authentication logic here
...
```

**With kube-authkit:**
```python
from kube_authkit import get_k8s_client, AuthConfig

class Client:
def __init__(
self,
host: Optional[str] = None,
namespace: str = "kubeflow",
auth_config: Optional[AuthConfig] = None,
# Deprecated legacy parameters
existing_token: Optional[str] = None,
**legacy_kwargs
):
# Simple unified authentication
if auth_config is None:
auth_config = AuthConfig(
k8s_api_host=host,
token=existing_token or legacy_kwargs.get('existing_token'),
)

self.api_client = get_k8s_client(auth_config)
self.namespace = namespace
```

### Kubeflow Training Client

```python
from kube_authkit import get_k8s_client, AuthConfig

class TrainingClient:
def __init__(
self,
config_file: Optional[str] = None,
context: Optional[str] = None,
auth_config: Optional[AuthConfig] = None,
):
if auth_config is None:
auth_config = AuthConfig(
kubeconfig_path=config_file,
# Auto-detects if kubeconfig_path is None
)

self.api_client = get_k8s_client(auth_config)
```

## Documentation Benefits

Kubeflow SDK can reference comprehensive kube-authkit documentation instead of maintaining separate docs:

- ✅ [Environment Variables Guide](https://github.com/opendatahub-io/kube-authkit/blob/main/ENVIRONMENT_VARIABLES.md)
- ✅ [Kubernetes Secrets Integration](https://github.com/opendatahub-io/kube-authkit/blob/main/ENVIRONMENT_VARIABLES.md#using-environment-variables-as-kubernetes-secrets)
- ✅ [Security Best Practices](https://github.com/opendatahub-io/kube-authkit/blob/main/ENVIRONMENT_VARIABLES.md#security-best-practices)
- ✅ [Authentication Strategy Reference](https://github.com/opendatahub-io/kube-authkit)
- ✅ [Troubleshooting Guide](https://github.com/opendatahub-io/kube-authkit/blob/main/ENVIRONMENT_VARIABLES.md#troubleshooting)

## Community Alignment

This aligns with broader Kubernetes ecosystem trends:

- **OpenDataHub** uses kube-authkit for notebook authentication
- **Standardized authentication patterns** across AI/ML platforms
- **Shared maintenance** reduces duplication across projects
- **Security improvements** benefit entire ecosystem

## Implementation Checklist

- [ ] Add kube-authkit as optional dependency
- [ ] Create authentication adapter layer
- [ ] Update Pipelines SDK to use kube-authkit
- [ ] Update Training SDK to use kube-authkit
- [ ] Update Katib SDK to use kube-authkit
- [ ] Update Model Registry SDK to use kube-authkit
- [ ] Write migration guide
- [ ] Update documentation
- [ ] Add integration tests
- [ ] Deprecate legacy authentication
- [ ] Remove legacy code (v2.0)

## Questions for Discussion

1. **Timeline**: What's a reasonable timeline for each migration phase?
2. **Backward Compatibility**: How long should we maintain legacy authentication?
3. **Optional vs Required**: Should kube-authkit be optional or required dependency?
4. **Version Compatibility**: What Kubeflow SDK versions should support this?
5. **Testing**: What additional test coverage is needed?

## References

- **kube-authkit Repository**: https://github.com/opendatahub-io/kube-authkit
- **Environment Variables Guide**: https://github.com/opendatahub-io/kube-authkit/blob/main/ENVIRONMENT_VARIABLES.md
- **Kubeflow Pipelines Authentication Docs**: https://www.kubeflow.org/docs/components/pipelines/user-guides/core-functions/connect-api/
- **CVE-2025-66418**: https://nvd.nist.gov/vuln/detail/CVE-2025-66418
- **CVE-2025-66471**: https://nvd.nist.gov/vuln/detail/CVE-2025-66471
- **Kubernetes Python Client 35.0.0**: https://pypi.org/project/kubernetes/35.0.0/

## Next Steps

If this proposal is accepted, I'm happy to:

1. Create a proof-of-concept PR for Pipelines SDK
2. Help write migration documentation
3. Collaborate on integration tests
4. Present at Kubeflow community meeting

---

**CC**: @kubeflow/kubeflow-sdk-team

Looking forward to community feedback on this proposal!