Passing xcoms as dictionary when using jinja templating to KubernetesPodOperator
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Description
Passing xcoms from upstream tasks to downstream KubernetesPodOperator's env_vars using jinja templating like, `env_vars="{{ task_instance.xcom_pull(task_ids='get_env_vars') }}",` fails as the variables are parsed as strings instead of dictionary.
It would be useful to have a feature that either allows kubernetespodoperator to parse the passed string as a dictionary using another parameter. Alternatively it would be nice to have the xcom pull be rendered as a dictionary by default.
### Use case/motivation
While it is still possible to achieve this using taskflow to load the variables once and use it several times, I would like to avoid having several relationship lines that are automatically created while using the taskflow api. For users like me using variables to store secrets to pass to pods as env, it would be useful to have this functionality to create elegant looking DAGs while at the same time not making unnecessary database calls during parse the DAG, thereby optimising the deployment.
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start with KubernetesPodOperator's env_vars handling and the Jinja templating path that processes task_instance.xcom_pull results. Reproduce the string-versus-dictionary behavior, then verify that the chosen feature makes dictionary-valued environment variables usable without the extra TaskFlow relationships described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- kubernetes, python
- Domain
- devops, infrastructure
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100