dbt-labs / dbt-labs/dbt-external-tables

Expand tests for Spark external tables

Open
#52 1 comment 0 reactions 0 assignees View on GitHub
enhancement spark Stale tech_debt
Dominant language
Jupyter Notebook
Stars
384
Forks
150
Avg merge
1h 25m
Merged PRs (30d)
1

Description

(split off from #10)

### Not currently tested
- [ ] ndjson (`s3://dbt-external-tables-testing/json/`)
- [ ] Hive-formatted external tables, since SQL Endpoint returns this error:
```
('42000', '[42000] [Simba][Hardy] (80) Syntax or semantic analysis error thrown in server while executing query. Error message from server: Error running query: org.apache.spark.sql.AnalysisException: Cannot use this data source. Only csv,json,avro,delta,parquet,orc,text data sources are supported on SQL Gateway.;;\nSubqueryAlias spark_catalog.dbt_jcohen.people_csv_unpartitioned_hive_format\n+- HiveTableRelation `dbt_jcohen`.`people_csv_unpartitioned_hive_format`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [id#2612, f\x00 (80) (SQLExecDirectW)')
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.