typelevel / typelevel/frameless
Encoder derivation for collection incompatible implementations for interpreted serde e.g. Seq instead of Vector
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 895
- Forks
- 135
- Avg merge
- 1d 16h
- Merged PRs (30d)
- 3
Description
Collection encoder uses MapObjects, this does not have special logic for Vector and will return a Seq that can be a different class.
Scope is serialisation with interpretation (many use-cases will do codegen), compilation of MapObjects is correct and uses builders. Eval of MapObjects with NewInstance fails as NewInstance will look for a Seq constructor parameter but only a Vector parameter exists.
In fixing #803 the use of Vector in UdfTests."one argument udf" failed as:
Message: scala.collection.immutable.Stream$Cons cannot be cast to scala.collection.immutable.Vector
which is true. The Stream$Cons is created by MapObjects:
private lazy val mapElements: scala.collection.Seq[_] => Any = customCollectionCls match {
case Some(cls) if classOf[WrappedArray[_]].isAssignableFrom(cls) =>
...
case Some(cls) if classOf[scala.collection.Seq[_]].isAssignableFrom(cls) =>
// Scala sequence
executeFuncOnCollection(_).toSeq
There are a number of these potential issues with all similar Seq derived types (Set derived as well).
The inbuilt ScalaReflection createIterableEncoder also correctly uses the appropriate builder.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the collection encoder's MapObjects evaluation and the failing Vector case in UdfTests."one argument udf". Compare the interpreted path with ScalaReflection's createIterableEncoder and inspect how NewInstance handles collection constructors. Done means interpreted serialization preserves the declared collection implementation, including Seq- and Set-derived types, while codegen remains correct.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100