derrickoswald / derrickoswald/CIMSpark

non-default database

Open
#18 2 comments 0 reactions 0 assignees View on GitHub
help wanted
Dominant language
Scala
Stars
19
Forks
1
PR merge metrics
No merged PRs in 30d

Description

The CIM RDD are currently added to the **default** SparkSQL database:
```sql
scala> spark.sql ("show databases").show
+------------+
|databaseName|
+------------+
| default|
+------------+
```

It would be good if the database could be specified.
- how to communicate the desired database to the CIMReader
- it would need to be [created](https://docs.databricks.com/spark/latest/spark-sql/language-manual/create-database.html) if it doesn't exist
- code in the CIMReader would need to [use the database](https://docs.databricks.com/spark/latest/spark-sql/language-manual/use-database.html) if it wasn't the default

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the CIMReader code, especially how it currently adds RDDs to SparkSQL's default database. Determine how the desired database should be communicated to CIMReader, then verify the work creates the database when needed and uses it before adding the RDDs. Done means CIM RDDs can be placed in a specified non-default database without breaking the default case.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala, spark
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.