json4s / json4s/json4s

CustomSerializer[Enumeration.Value] clobbers EnumNameSerializer()

Open
#712 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Scala
Stars
1.5k
Forks
324
Avg merge
4h 35m
Merged PRs (30d)
30

Description

Context

Hi, I have kind of an unusual edge case. I recently had to change an enum class to be backwards-compatible with some legacy code, but expanded and renamed new terms. This works entirely fine and is not the focus, but what surfaced this issue.

Previously, I could (de)serialize just fine with these Formats:

implicit val formats = DefaultFormats + new EnumNameSerializer(DayOfWeek)

But since the change I had to define a CustomSerializer

implicit val formats = DefaultFormats + DayOfWeekSerializer
class DayOfWeekSerializer extends CustomSerializer[DayOfWeek.Value](format => (
  { case Jstring(value) => DayOfWeek.customParse(value) },
  { case v: DayOfWeek.Value => JString(v.toString) }
))
object DayOfWeek extends Enumeration {
  val Monday, Tuesday, Wednesday, Thursday, Friday, Saturday, Sunday = Value

  /*
   * We used to only store abbreviated weekdays, so now we have to handle parsing them
   * If it's not one of our special cases, we can rely on the default behavior.
   * Note we cannot override withName since it is final. Found that out the hard way.
   */  
  def customParse(value: String): Value = {
    value match {
      case "Mon" => Monday
      case "Tue" => Tuesday
      case "Wed" => Wednesday
      case "Thu" => Thursday
      case "Fri" => Friday
      case _ => values.withName(value)
    }
  }
}

That works just fine... until I add another Enum into the mix. This breaks, full stack trace below.

implicit val formats = DefaultFormats + new EnumNameSerializer(Color) + new DayOfWeekSerializer
Attempted workarounds:
  • Add conditional to the case in the DayOfWeekSerializer
    { case JString(value) if Try(DayOfWeek.customParse(value)).isSuccess => DayOfWeek.customParse(value) }
    Does not work, it seems the conditional is ignored completely
  • Swap order of formats
    implicit val formats = DefaultFormats + new DayOfWeekSerializer + new EnumNameSerializer(Color)
    swapping the order of the serializers works and the code runs as expected without error. That being said, I'd rather not have to rely on ordering because it's not immediately clear.
Ask

I think the ordering of Formats should not matter, though I will admit my understanding of the depths of json4s is not thorough. From my limited perspective as a user, I would expect ordering to not be relevant. Or at least, even if ordering is relevant, it won't break other formats. I do get that Enums are a difficult edge case since they carry no class data at runtime.

If there's something more/different/better I could be doing with the CustomSerializer implementation please let me know. (I can also cross-post to StackOverflow if you prefer)

Stacktrace
[error]  org.json4s.package$MappingException: No usable value for color
[error]  Can't convert JString(GREEN) to class scala.Enumeration$Value (package.scala:95)
[error] org.json4s.reflect.package$.fail(package.scala:95)
[error] org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:548)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:572)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.instantiate(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:630)
[error] org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:416)
[error] org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
[error] org.json4s.Extraction$.customOrElse(Extraction.scala:637)
[error] org.json4s.Extraction$.extract(Extraction.scala:408)
[error] org.json4s.Extraction$.extract(Extraction.scala:40)
[error] org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
[error] org.json4s.native.Serialization$.read(Serialization.scala:71)
[error] org.json4s.Serialization.read(Serialization.scala:25)
[error] org.json4s.Serialization.read$(Serialization.scala:25)
[error] org.json4s.native.Serialization$.read(Serialization.scala:32)
[error] /* three lines of application code */
[error] org.json4s.CustomSerializer$$anonfun$deserialize$3.applyOrElse(Formats.scala:485)
[error] org.json4s.CustomSerializer$$anonfun$deserialize$3.applyOrElse(Formats.scala:482)
[error] org.json4s.Extraction$.customOrElse(Extraction.scala:637)
[error] org.json4s.Extraction$.extract(Extraction.scala:408)
[error] org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:534)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:572)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.instantiate(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:630)
[error] org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:416)
[error] org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
[error] org.json4s.Extraction$.customOrElse(Extraction.scala:637)
[error] org.json4s.Extraction$.extract(Extraction.scala:408)
[error] org.json4s.Extraction$.extract(Extraction.scala:40)
[error] org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
[error] org.json4s.native.Serialization$.read(Serialization.scala:71)
[error] org.json4s.Serialization.read(Serialization.scala:25)
[error] org.json4s.Serialization.read$(Serialization.scala:25)
[error] org.json4s.native.Serialization$.read(Serialization.scala:32)
[error] /* same 3 lines again */
[error] CAUSED BY
[error]  org.json4s.package$MappingException: Can't convert JString(GREEN) to class scala.Enumeration$Value (Formats.scala:485)
[error] org.json4s.CustomSerializer$$anonfun$deserialize$3.applyOrElse(Formats.scala:485)
[error] org.json4s.CustomSerializer$$anonfun$deserialize$3.applyOrElse(Formats.scala:482)
[error] org.json4s.Extraction$.customOrElse(Extraction.scala:637)
[error] org.json4s.Extraction$.extract(Extraction.scala:408)
[error] org.json4s.Extraction$ClassInstanceBuilder.org$json4s$Extraction$ClassInstanceBuilder$$buildCtorArg(Extraction.scala:534)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:572)
[error] org.json4s.Extraction$ClassInstanceBuilder$$anonfun$3.applyOrElse(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.instantiate(Extraction.scala:570)
[error] org.json4s.Extraction$ClassInstanceBuilder.result(Extraction.scala:630)
[error] org.json4s.Extraction$.$anonfun$extract$10(Extraction.scala:416)
[error] org.json4s.Extraction$.$anonfun$customOrElse$1(Extraction.scala:637)
[error] org.json4s.Extraction$.customOrElse(Extraction.scala:637)
[error] org.json4s.Extraction$.extract(Extraction.scala:408)
[error] org.json4s.Extraction$.extract(Extraction.scala:40)
[error] org.json4s.ExtractableJsonAstNode.extract(ExtractableJsonAstNode.scala:21)
[error] org.json4s.native.Serialization$.read(Serialization.scala:71)
[error] org.json4s.Serialization.read(Serialization.scala:25)
[error] org.json4s.Serialization.read$(Serialization.scala:25)
[error] org.json4s.native.Serialization$.read(Serialization.scala:32)
[error] /* same 3 lines, entry point */
json4s version

3.6.+

scala version

2.12.6

jdk version

OpenJdk 1.8.0

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the two Formats orderings shown in the issue, then start at CustomSerializer in Formats.scala around lines 482-485 and follow the Extraction.scala customOrElse path from the stack trace. Compare how EnumNameSerializer and DayOfWeekSerializer are selected, and define completion as equivalent behavior regardless of serializer ordering, with regression coverage for both orderings.

Written by the indexing model from the issue text.

Assessment

Tech stack
scala
Domain
backend-api-design
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.