typelevel / typelevel/frameless

Encoder derivation for collection incompatible implementations for interpreted serde e.g. Seq instead of Vector

Open
#804 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
895
Forks
135
Avg merge
1d 16h
Merged PRs (30d)
3

Description

Collection encoder uses MapObjects, this does not have special logic for Vector and will return a Seq that can be a different class.

Scope is serialisation with interpretation (many use-cases will do codegen), compilation of MapObjects is correct and uses builders. Eval of MapObjects with NewInstance fails as NewInstance will look for a Seq constructor parameter but only a Vector parameter exists.

In fixing #803 the use of Vector in UdfTests."one argument udf" failed as:

Message: scala.collection.immutable.Stream$Cons cannot be cast to scala.collection.immutable.Vector

which is true. The Stream$Cons is created by MapObjects:

  private lazy val mapElements: scala.collection.Seq[_] => Any = customCollectionCls match {
    case Some(cls) if classOf[WrappedArray[_]].isAssignableFrom(cls) =>
      ...
    case Some(cls) if classOf[scala.collection.Seq[_]].isAssignableFrom(cls) =>
      // Scala sequence
      executeFuncOnCollection(_).toSeq

There are a number of these potential issues with all similar Seq derived types (Set derived as well).

The inbuilt ScalaReflection createIterableEncoder also correctly uses the appropriate builder.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the collection encoder's MapObjects evaluation and the failing Vector case in UdfTests."one argument udf". Compare the interpreted path with ScalaReflection's createIterableEncoder and inspect how NewInstance handles collection constructors. Done means interpreted serialization preserves the declared collection implementation, including Seq- and Set-derived types, while codegen remains correct.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.