Airflow not publishing logs to Azure App Insights
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 9h
- Merged PRs (30d)
- 472
Description
### Under which category would you file this issue?
Airflow Core
### Apache Airflow version
3.2.2
### What happened and how to reproduce it?
Upon migrating to airflow 3.2.2 from 2.9.3 my airflow application stopped publishing logs to Azure App Insights. Running Airlfow on AKS.
For 2.9.3 version I had written a custom handler and attached it to the airflow.task logger and that enabled the tunnel to push the airflow task logs to Azure app Insights in the CustomEvents table
Now using the similar approach for 3.2.2, the logs are not being pushed to AppInsights.
Library used for AppInsights connection: azure-monitor-opentelemetry==1.8.9, azure-monitor-opentelemetry-exporter==1.0.0b55.
I tried putting in logs for the handler initialisation- the initialsation logs were shown but the emit() logs never showed up in any of the pods
### What you think should happen instead?
_No response_
### Operating System
_No response_
### Deployment
None
### Apache Airflow Provider(s)
_No response_
### Versions of Apache Airflow Providers
_No response_
### Official Helm Chart version
Not Applicable
### Kubernetes Version
_No response_
### Helm Chart configuration
_No response_
### Docker Image customizations
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
No files or tests are named. Start by comparing the custom handler and airflow.task logger behavior between Airflow 2.9.3 and 3.2.2, then inspect pod logs to determine why initialization appears but emit() does not; done means task logs arrive in Azure App Insights CustomEvents.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, kubernetes, python
- Domain
- cloud, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 38/100