derrickoswald / derrickoswald/CIMSpark
non-default database
- Dominant language
- Scala
- Stars
- 19
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
The CIM RDD are currently added to the **default** SparkSQL database:
```sql
scala> spark.sql ("show databases").show
+------------+
|databaseName|
+------------+
| default|
+------------+
```
It would be good if the database could be specified.
- how to communicate the desired database to the CIMReader
- it would need to be [created](https://docs.databricks.com/spark/latest/spark-sql/language-manual/create-database.html) if it doesn't exist
- code in the CIMReader would need to [use the database](https://docs.databricks.com/spark/latest/spark-sql/language-manual/use-database.html) if it wasn't the default
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the CIMReader code, especially how it currently adds RDDs to SparkSQL's default database. Determine how the desired database should be communicated to CIMReader, then verify the work creates the database when needed and uses it before adding the RDDs. Done means CIM RDDs can be placed in a specified non-default database without breaking the default case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- databases
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100