Unable to specify ssl custom CA for operator pod using KubernetesPodOperator
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
Other Airflow 2 version (please specify below)
### If "Other Airflow 2 version" selected, which one?
2.10.5
### What happened?
We are running Airflow in our own Kubernetes cluster, which uses an internal intermediate Certificate Authority (CA) for service-to-service communication.
Due to a known limitation of urllib3 (used by Python), which does not handle intermediate root certificates properly, we explicitly provide the full certificate chain for Airflow and other components. In most cases, this is resolved by passing the complete certificate bundle via an environment variable, and pointing relevant Airflow components to it using standard ENV configuration. This works correctly outside the Kubernetes executor.
However, when using the KubernetesPodOperator with Kubernetes Executor, the overridden certificate (provided via ENV) is ignored. This results in an SSL validation error during pod spawn. We could not find any documented way — either in the official docs or through experimentation — to explicitly pass this custom CA to the pods launched by the KubernetesPodOperator spawner pod.
### What you think should happen instead?
There should be a supported and documented method to pass additional CA certificates (e.g. via ENV or config) to the operator pods spawned by the Kubernetes Executor, similar to how it works for other Airflow components.
### How to reproduce
The pods launched by the Kubernetes Executor do not inherit or respect the custom CA provided via ENV, leading to SSL errors during execution.
Configure Airflow to use a custom CA bundle via ENV.
Use the Kubernetes Executor with KubernetesPodOperator.
Observe that spawned pods fail with SSL-related errors due to untrusted intermediate CA.
### Operating System
k8s 1.30.10
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Environment:
Airflow version: 2.10.5
K8s: 1.30.10
Certs: custom fullchain
Executor: Kubernetes Executor
Operator: KubernetesPodOperator
### Anything else?
Error:
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='10.233.0.1', port=443): Max retries exceeded with url: /api/v1/namespaces/airflow/pods?labelSelector=dag_id%3Dcifar100_classification%2Ckubernetes_pod_operator%3DTrue%2Crun_id%3Dmanual__2025-07-10T132839.5223690000-4c7fe4717%2Ctask_id%3Dtrain_model%2Calready_checked%21%3DTrue%2C%21airflow-worker (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get issuer certificate (_ssl.c:1010)')))
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start by tracing how KubernetesPodOperator pods launched through the Kubernetes Executor receive environment and certificate settings, using the reported custom CA configuration and SSL failure to reproduce the issue. Done means there is a supported, documented way for spawned operator pods to use the additional CA bundle and the reported certificate validation error no longer occurs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- helm, kubernetes, python
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100