New fields added in certain situations

In a JSON file, if you use a compound field's name (for example "authorName") outside of the compound structure to create a field, and use that JSON to create a new dataset via the Native API, it will create a new field at a higher level in the metadata (outside the compound it's a part of).

Note: I tried testing out the reverse, by adding a non-compound field (title) to a compound one (author) and that returned an error.

What steps does it take to reproduce the issue?
Add data like this to a dataset you're creating:

  "datasetVersion": {
    "metadataBlocks": {
      "citation": {
        "displayName": "Citation Metadata",
        "name": "citation",
        "fields": [
          {
            "typeName": "newField",
            "multiple": false,
            "typeClass": "primitive",
            "value": "This is a fun and new field that's not compound, I just made it up!"
          },
          {
            "typeName": "authorName",
            "multiple": false,
            "typeClass": "primitive",
            "value": "This is a fun and new field, I just made it up!"
          },
          {
            "typeName":"otherGeographicCoverage",
            "multiple":false,
            "typeClass":"primitive",
            "value":"Asia and Pacific Islands"
          },
...

Notice that this does three things:

• tries to add a totally new field -- this will fail
• tries to add a "Name" for an author outside of the author compound field -- this will succeed.
• tries to add "otherGeographicCoverage" inside the "Citations" metadata block -- this does end up adding the field, inside the Geospatial Block.

Then add the datasets, with or without the "doNotValidate" flag, as that doesn't seem to matter:

curl -H "X-Dataverse-key:$TOKEN" -X POST "https://dataverse-test.yale.edu/api/dataverses/yale/datasets" --upload-file my_test.json -H 'Content-type:application/json'

You'll now see:

• To whom does it occur (all users, curators, superusers)?

I'm a superuser admin, but I imagine it would happen to anyone adding datasets via the API.

• What did you expect to happen?
  I'd expected the extra fields to trigger an error or be ignored, as they are in the wrong spots.

Which version of Dataverse are you using?

5.14 and 6.6 - I was able to reproduce this in both setups

Any related open or closed issues to this bug report?

Are you thinking about creating a pull request for this issue?

No, unfortunately I'm not setup to try to fix it. I suspect it has to do with how fields are added -- that it's adding the data for a field, but not checking the context of where the data is being added (ie, is it in the compound field it's a part of?).

Start by reproducing the dataset creation request against the Native API with the supplied metadataBlocks JSON, both with and without doNotValidate. Trace how typeName values are resolved across metadata blocks and compound fields, then verify that misplaced compound-field names and fields from another block are rejected or ignored while valid fields continue to work.