Support disabling of dag_dir_list_interval
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 483
Description
### Apache Airflow version
2.5.0
### What happened
We have a unique situation with our Airflow deployment, in that we bake the DAGs into the image pre-deployment, meaning that a deployment is triggered every time there is a new DAG contribution.
During a recent performance troubleshooting, we have uncovered that the scheduler consumes much more memory than it should, and while looking for ways to optimize it, we came across the `dag_dir_list_interval` configuration. It's currently set to the default of 5 minutes. However, it seems like there is no possibility to disable it barring the possibility of setting it to a really high value like 86400.
### What you think should happen instead
It would be nice if the parameter supports a value, say `-1 `that disables the scan altogether.
### How to reproduce
Default configuration, the scan takes place every 5 minutes.
### Operating System
Debian GNU/Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Official Apache Airflow Helm Chart
### Deployment details
Airflow deployed with official helm chart on AKS.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Assessment
This issue has not been assessed yet.