elastic / elastic/package-spec

[Change Proposal] A way to declare a field that is usually a scalar but sometimes an object

Open
#1,211 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
20
Forks
93
Avg merge
15h 10m
Merged PRs (30d)
12

Description

# [Change Proposal] A way to declare a field that is usually a scalar but sometimes an object

> Draft for **elastic/package-spec**. Type: `[Change Proposal]`.
> Companion validator bug: **elastic/elastic-package#3788**.
> Motivating PR: elastic/integrations#20306

## The problem

Some sources send one field as either a scalar or an object, depending on the
event. The spec has no way to *declare* that. Authors can make the mapping work
with `subobjects: false` + `object_type`, but that's implicit (nothing says the
field is polymorphic) and tooling doesn't accept the object shape.

## Example

GitHub audit `repository_ruleset.*` events send
`ruleset_conditions[].parameters.{include,exclude,repository_ids}`. Usually these
are arrays of repository-name strings. For the `repository_property` target,
GitHub sends the same keys as objects instead:

```json
{ "name": "example-property", "source": "custom", "property_values": ["true"] }
```

So `…parameters.include` is legitimately either:

- `["repo-a", "repo-b"]` — keyword array, or
- `{ "name": "...", "source": "...", "property_values": ["..."] }` — object.

Before PR #20306 the object shape collided with the `keyword` mapping and went to
the failure store. #20306 fixes it with:

```yaml
- name: parameters
type: object
subobjects: false
object_type: keyword
object_type_mapping_type: "*"
- name: parameters.exclude
type: keyword
- name: parameters.include
type: keyword
- name: parameters.repository_ids
type: keyword
```

This is ES-valid: the string form stays statically mapped, and the object form
flattens to dynamic keyword leaves (`parameters.include.name`, etc.). `okta` uses
the same idiom, so it's established — just undocumented.

## What the spec can and can't express today

- Can: a scalar (`type: keyword`).
- Can: an object of scalars (`type: object` + `object_type: keyword` +
`subobjects: false`).
- Can't: "this field is a scalar that may also be an object of scalars."

The `subobjects: false` trick reaches the right ES mapping, but the intent is
invisible to readers, docs-gen, and validators — and the validator rejects the
object shape outright, so it can't be tested (companion issue).

## Proposal (for discussion)

Give authors an explicit way to declare a scalar-or-object field. Two options:

1. A field attribute (e.g. `allow_object_values: true`) on a scalar field,
meaning "scalar, or an object whose leaves are this type," that maps to
`subobjects: false` + `object_type` internally.
2. Or formally document the `subobjects: false` + dotted-sibling pattern as the
sanctioned idiom, and require validators to accept object values for it.

Goal either way: make the intent explicit in `fields.yml`, and make the object
shape testable.

## Related

- Motivating PR: elastic/integrations#20306
- Companion validator bug: elastic/elastic-package#3788

Contributor guide

Open the contributing guide

Research direction

Start by reading the fields.yml patterns described in the proposal, especially the subobjects: false and dotted-sibling example, then review companion validator issue elastic/elastic-package#3788 and motivating PR elastic/integrations#20306. Compare the proposed field attribute with formally documenting the existing idiom. Done means the chosen declaration is explicit in the spec and the object shape is accepted and testable by validators.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.