Complex DataTypes in LeapFrames
- Dominant language
- Scala
- Stars
- 1.5k
- Forks
- 315
- PR merge metrics
- No merged PRs in 30d
Description
I am new to both spark and MLeap and am working in a Java codebase trying to update our prediction service to use MLeap (v0.12.0) to predict on spark (v2.3.1) models.
Part of that requires recreating spark DataFrames as MLeap frames. In the spark frames we have a number of complex types: `MapType`s, `ArrayType[ArrayType[String]`s , and `ArrayType[StructType]`s.
If these are supported by MLeap, I can't seem to understand how to recreate them. Any advice would be greatly appreciated!
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by checking MLeap v0.12.0's frame and schema support for MapType, nested ArrayType, and ArrayType[StructType], then compare it with the Spark 2.3.1 types described in the issue. Done means documenting how supported types can be recreated or confirming that missing support would require a larger feature.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, scala
- Domain
- data, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100