Invalid input for optional fields should result in a MappingException, regardless of strictOptionParsing

Hi there,

We've been wanting to update to json4s 3.7.x or 4.x for quite some time, but are currently blocked by the way strictOptionParsing is implemented as of #688 .

We use json4s together with akka-http and elastic4s.
In our customer-facing API:

• We don't want to require our customers to spell out all optional properties that they don't use
• We want to return a BadRequest if they provide an incorrect enum value (we use enumeraturm), for instance sipped instead of shipped

When retrieving data from Elasticsearch:

• If an optional field is added; it should default to None for existing records (that don't have this field), due to Backwards Compatibility

The behavior we seek:

• When an optional field is not provided, it should default to None
• When an optional field is provided but an invalid value is provided: throw an exception.

This is the behavior in 3.6.12 when using withStrictOptionParsing, however as of 3.7 and above:

• using withStrictOptionParsing requires to write out all optional fields
• not using withStrictOptionParsing allows you to provide an incorrect value for an optional field.

I was wondering if you'd appreciate a PR for a patch for when strictOptionParsing = false to change the behavior to what is described above.

Example of parsing an Option[BigDecimal]:

JSON	Result
empty	None
null	None
12.5	Some(12.5)
"Hello there"	MappingException

json4s version

3.6.12

scala version

2.13.10

jdk version

openjdk 11.0.17 2022-10-18

Minimal example

import org.json4s.DefaultFormats
import org.json4s.jackson.Serialization

object Example extends App {

  case class RootClass(sub: Option[SubClass])
  case class SubClass(value: Option[BigDecimal])

  implicit val formats = DefaultFormats

  /*
   * SubClass examples
   */

  val emptySubJson = """{}"""
  println(s"json=${emptySubJson}, result=${Serialization.read[SubClass](emptySubJson)}") // Expected: SubClass(None)

  val correctSubJson = """{"value": 12.5}"""
  println(s"json=${correctSubJson}, result=${Serialization.read[SubClass](correctSubJson)}") // Expected: SubClass(Some(12.5))

  val incorrectSubJson = """{"value": "hello there"}"""
  println(s"json=${incorrectSubJson}, result=${Serialization.read[SubClass](incorrectSubJson)}") // Expected: Exception

  /*
   * RootClass examples
   */

  val emptyRootJson = """{}"""
  println(s"json=${emptyRootJson}, result=${Serialization.read[RootClass](emptyRootJson)}") // Expected: RootClass(None)

  val correctRootJson = """{"sub": {"value": 12.5}}"""
  println(
    s"json=${correctRootJson}, result=${Serialization.read[RootClass](correctRootJson)}"
  ) // Expected: RootClass(Some(SubClass(Some(12.5))))

  val incorrectRootJson = """{"sub": {"value": "hello there"}}"""
  println(s"json=${incorrectRootJson}, result=${Serialization.read[RootClass](incorrectRootJson)}") // Expected: Exception
}

Generated output:

json={}, result=SubClass(None)
json={"value": 12.5}, result=SubClass(Some(12.5))
json={"value": "hello there"}, result=SubClass(None)
json={}, result=RootClass(None)
json={"sub": {"value": 12.5}}, result=RootClass(Some(SubClass(Some(12.5))))
json={"sub": {"value": "hello there"}}, result=RootClass(Some(SubClass(None)))

Start by running the minimal Scala example and compare the missing, null, valid, and invalid optional-field cases under both strictOptionParsing settings. Trace the option parsing behavior responsible for invalid values being treated as absent. Done means missing or null fields still produce None, while supplied invalid values produce a MappingException.