byzer-org / byzer-org/byzer-shell
Byzer Shell 默认配置文件复用 $BYZER_HOME/conf/byzer.properties
- Dominant language
- Rust
- Stars
- 6
- Forks
- 2
- PR merge metrics
- No merged PRs in 30d
Description
当前版本默认的配置文件是执行 `byzer-shell` 的目录下的 `.mlsql.config` 文件,这个和产品包的体系内容不搭,需要在 byzer-shell 能够以 `$BYZER_HOME/conf/byzer.properties` 和 `$BYZER_HOME/conf/byzer.properties.override` 文件来控制配置。
配置大概区分为三个部分
```
java.home=/Users/allwefantasy/projects/mlsql/dist/mlsql-lang/jdk8
engine.memory=6048m
# engine.url=http://127.0.0.1:9004/
# engine.url=http://127.0.0.1:9004/
engine.spark.sql.extensions=io.delta.sql.DeltaSparkSessionExtension
engine.spark.sql.catalog.spark_catalog=org.apache.spark.sql.delta.catalog.DeltaCatalog
engine.spark.shuffle.spill.batchSize=1000
engine.spark.shuffle.spill.numElementsForceSpillThreshold=10000
engine.spark.sql.shuffle.partitions=8
engine.spark.shuffle.accurateBlockThreshold=5242880
engine.spark.memory.fraction=0.1
engine.spark.memory.storageFraction=0.1
engine.spark.sql.inMemoryColumnarStorage.batchSize=1000
engine.spark.sql.files.maxPartitionBytes=5242880
engine.spark.sql.parquet.columnarReaderBatchSize=1000
engine.spark.sql.pivotMaxValues=1000
engine.spark.unsafe.sorter.spill.reader.buffer.size=1m
engine.spark.file.transferTo=false
engine.spark.shuffle.file.buffer=1m
engine.spark.shuffle.unsafe.file.ouput.buffer=1m
engine.spark.io.compression.lz4.blockSize=128k
user.owner=admin
user.access_token=123
user.outputSize=100
```
Contributor guide
Assessment
This issue has not been assessed yet.