4paradigm / 4paradigm/OpenMLDB
Fail to get table constrains of Hive tables with Spark API
- Ngôn ngữ chính
- C++
- Star
- 1.7k
- Fork
- 331
- Merge trung bình
- 12 ngày 12 giờ
- Pull request đã merge (30 ngày)
- 1
Mô tả
Now we can not get table constrains like `not null` with Spark API. It may be the bug of Spark.
Here is the process to re-produce the issue.
We can create the table with constrains with hive.
```
create table notnull_col_table2 (col1 int not null, col2 int);
```
But we may not get the not null constrains with Spark API.
```
def main(argv: Array[String]): Unit = {
val spark = SparkSession.builder()
.master("local")
.config("spark.hadoop.hive.metastore.uris", "thrift://localhost:9083")
.config("spark.sql.catalogImplementation", "hive")
.getOrCreate()
val df = spark.table("db1.notnull_col_table2")
val schema = df.schema
println(schema) # StructType(StructField(col1,IntegerType,true),StructField(col2,IntegerType,true))
val col1_schema = schema.fields(0)
println(col1_schema.nullable) # true
}
```
Hướng dẫn đóng góp
Đánh giá
Issue này chưa được đánh giá.