[CH] Fix GlutenClickHouseMergeTreeWriteOnHDFSSuite and GlutenClickHouseMergeTreeWriteOnS3Suite after gluten domain name changes
- Dominant language
- Scala
- Stars
- 1.6k
- Forks
- 657
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 80
Description
### Backend
CH (ClickHouse)
### Bug description
[2024-03-29T12:02:49.115Z] Caused by: org.apache.gluten.exception.GlutenException: Unknown storage policy `__s3_main`
[2024-03-29T12:02:49.115Z] 0. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Common/Exception.cpp:96: DB::Exception::Exception(DB::Exception::MessageMasked&&, int, bool) @ 0x000000000b1546bb
[2024-03-29T12:02:49.115Z] 1. ../ClickHouse/contrib/llvm-project/libcxx/include/string:1499: DB::Exception::Exception(int, FormatStringHelperImpl::type>, String&&) @ 0x000000000584c9a3
[2024-03-29T12:02:49.115Z] 2. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Disks/StoragePolicy.cpp:0: DB::StoragePolicySelector::get(String const&) const @ 0x000000000e2ecb2b
[2024-03-29T12:02:49.115Z] 3. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Interpreters/Context.cpp:0: DB::Context::getStoragePolicy(String const&) const @ 0x000000000e2bf582
[2024-03-29T12:02:49.115Z] 4. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../ClickHouse/src/Storages/MergeTree/MergeTreeData.cpp:0: DB::MergeTreeData::getStoragePolicy() const @ 0x000000000f2ffe8b
[2024-03-29T12:02:49.115Z] 5. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Storages/Mergetree/SparkMergeTreeWriter.cpp:183: local_engine::SparkMergeTreeWriter::writeTempPart(DB::BlockWithPartition&, std::shared_ptr const&) @ 0x000000000b51d4df
[2024-03-29T12:02:49.115Z] 6. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/Storages/Mergetree/SparkMergeTreeWriter.cpp:79: local_engine::SparkMergeTreeWriter::finalize() @ 0x000000000b51c720
[2024-03-29T12:02:49.115Z] 7. /home/jenkins/agent/workspace/gluten/gluten-ci/gluten/cpp-ch/build/../local-engine/local_engine_jni.cpp:0: Java_org_apache_spark_sql_execution_datasources_CHDatasourceJniWrapper_closeMergeTreeWriter @ 0x0000000005bf235f
[2024-03-29T12:02:49.115Z]
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.CHDatasourceJniWrapper.closeMergeTreeWriter(Native Method)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeOutputWriter.close(MergeTreeOutputWriter.scala:50)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeFileFormatDataWriter.releaseCurrentWriter(MergeTreeFileFormatDataWriter.scala:72)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeBaseDynamicPartitionDataWriter.renewCurrentWriter(MergeTreeFileFormatDataWriter.scala:304)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeDynamicPartitionDataSingleWriter.beforeWrite(MergeTreeFileFormatDataWriter.scala:429)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeDynamicPartitionDataSingleWriter.write(MergeTreeFileFormatDataWriter.scala:449)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeFileFormatDataWriter.writeWithMetrics(MergeTreeFileFormatDataWriter.scala:97)
[2024-03-29T12:02:49.115Z] at org.apache.spark.sql.execution.datasources.v1.clickhouse.MergeTreeFileFormatDataWriter.writeWithIterator(MergeTreeFileFormatDataWriter.scala:104)
### Spark version
None
### Spark configurations
_No response_
### System information
_No response_
### Relevant logs
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.