[CH]put clickhouse's log under yarn container's log path
Open
enhancement
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Description
Now we set clickhouse log file path by setting
``` spark.gluten.sql.columnar.backend.ch.runtime_config.logger.log ```
and
``` spark.gluten.sql.columnar.backend.ch.runtime_config.logger.errorlog ```
to an absolute path.
In YARN mode, we can put those logs under each container's log path. So we can easilly view them via spark UI.
Contributor guide
Research direction
Start by tracing where spark.gluten.sql.columnar.backend.ch.runtime_config.logger.log and errorlog are consumed when running in YARN mode. Confirm the ClickHouse logs are written under each YARN container's log path and can be viewed through the Spark UI; the issue names no specific files or tests.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- clickhouse
- Domain
- backend, observability
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100