elastic / elastic/package-spec
[Change Proposal] A way to declare a field that is usually a scalar but sometimes an object
- Dominant language
- Go
- Stars
- 20
- Forks
- 93
- Avg merge
- 15h 10m
- Merged PRs (30d)
- 12
Description
# [Change Proposal] A way to declare a field that is usually a scalar but sometimes an object
> Draft for **elastic/package-spec**. Type: `[Change Proposal]`.
> Companion validator bug: **elastic/elastic-package#3788**.
> Motivating PR: elastic/integrations#20306
## The problem
Some sources send one field as either a scalar or an object, depending on the
event. The spec has no way to *declare* that. Authors can make the mapping work
with `subobjects: false` + `object_type`, but that's implicit (nothing says the
field is polymorphic) and tooling doesn't accept the object shape.
## Example
GitHub audit `repository_ruleset.*` events send
`ruleset_conditions[].parameters.{include,exclude,repository_ids}`. Usually these
are arrays of repository-name strings. For the `repository_property` target,
GitHub sends the same keys as objects instead:
```json
{ "name": "example-property", "source": "custom", "property_values": ["true"] }
```
So `…parameters.include` is legitimately either:
- `["repo-a", "repo-b"]` — keyword array, or
- `{ "name": "...", "source": "...", "property_values": ["..."] }` — object.
Before PR #20306 the object shape collided with the `keyword` mapping and went to
the failure store. #20306 fixes it with:
```yaml
- name: parameters
type: object
subobjects: false
object_type: keyword
object_type_mapping_type: "*"
- name: parameters.exclude
type: keyword
- name: parameters.include
type: keyword
- name: parameters.repository_ids
type: keyword
```
This is ES-valid: the string form stays statically mapped, and the object form
flattens to dynamic keyword leaves (`parameters.include.name`, etc.). `okta` uses
the same idiom, so it's established — just undocumented.
## What the spec can and can't express today
- Can: a scalar (`type: keyword`).
- Can: an object of scalars (`type: object` + `object_type: keyword` +
`subobjects: false`).
- Can't: "this field is a scalar that may also be an object of scalars."
The `subobjects: false` trick reaches the right ES mapping, but the intent is
invisible to readers, docs-gen, and validators — and the validator rejects the
object shape outright, so it can't be tested (companion issue).
## Proposal (for discussion)
Give authors an explicit way to declare a scalar-or-object field. Two options:
1. A field attribute (e.g. `allow_object_values: true`) on a scalar field,
meaning "scalar, or an object whose leaves are this type," that maps to
`subobjects: false` + `object_type` internally.
2. Or formally document the `subobjects: false` + dotted-sibling pattern as the
sanctioned idiom, and require validators to accept object values for it.
Goal either way: make the intent explicit in `fields.yml`, and make the object
shape testable.
## Related
- Motivating PR: elastic/integrations#20306
- Companion validator bug: elastic/elastic-package#3788
Contributor guide
Research direction
Start by reading the fields.yml patterns described in the proposal, especially the subobjects: false and dotted-sibling example, then review companion validator issue elastic/elastic-package#3788 and motivating PR elastic/integrations#20306. Compare the proposed field attribute with formally documenting the existing idiom. Done means the chosen declaration is explicit in the spec and the object shape is accepted and testable by validators.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100