typelevel / typelevel/frameless
The right way to convert a column ?
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 895
- Forks
- 135
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 3
Description
Hello,
I am starting with Frameless and I am having a hard time converting my code based on spark-Dataframes to the Frameless framework.
The blocking point I reach now is how to override a column.
Let's say I have a dataframe with col1,col2,..., myColumn .
myColumn is a String and it was an export from a database where this column is actually a Seq[String] so I now need to convert it back to it's type.
I used to do
df
.withColumn("myColumn", toArray($"myColumn")
How would you do the same thing with Frameless ? Do you need two case class ? And use withColumnTuple and dropTuple ?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No file or test is named. Start by reviewing Frameless's column transformation APIs, especially withColumnTuple and dropTuple, alongside the Spark DataFrame equivalent described in the issue. Done would be a clear, documented recommendation for converting the string column to Seq[String], including whether an additional case class is needed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala, spark
- Domain
- data
- Issue type
- Documentation
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 15/100