limit print log of SQL statement
- Dominant language
- Python
- Stars
- 46.9k
- Forks
- 17.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 484
Description
### Description
The print log _self.log.info("Running statement: %s, parameters: %s", sql_statement, parameters)_ case the UI to lag is a case where the sql statement is long e.g insert statement with 150000 row
https://github.com/apache/airflow/blob/569599ab4fc6a3c7b6e34e23bfea65bb50551c1f/providers/common/sql/src/airflow/providers/common/sql/hooks/sql.py#L810C68-L810C81
**Feature Idea**: Reduce the sql_statement string length to 1000 characters or something not to large
### Use case/motivation
When **sql_statement** is large, like an insert statement with 1 million rows, the UI will not lag when viewing logs and also this will reduce the size of the log file
### Related issues
_No response_
### Are you willing to submit a PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
Contributor guide
Research direction
Start in providers/common/sql/src/airflow/providers/common/sql/hooks/sql.py at the SQL statement logging call identified in the issue. Determine how to cap the logged SQL text while preserving useful parameter and statement context. Done means very large statements no longer produce unbounded log output, while ordinary statements remain readable.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, sql
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100