elastic / elastic/elastic-package

Pipeline validator rejects object values under `subobjects: false` + `object_type`, though ES accepts them

Open
#3,788 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
72
Forks
141
Avg merge
19h 42m
Merged PRs (30d)
55

Description

# Pipeline validator rejects object values under `subobjects: false` + `object_type`, though ES accepts them

> Draft for **elastic/elastic-package** (bug). Companion design proposal:
> **elastic/package-spec#1211**. Motivating PR: elastic/integrations#20306.
> Filed separately so the concrete validator behavior is actionable; may be
> resolved as part of the package-spec discussion.

## The problem

With a field declared `type: object` + `subobjects: false` + `object_type:
keyword`, Elasticsearch indexes a value that is either a keyword (array) or an
object of keywords — the object is flattened to dotted keyword leaves. The
`elastic-package` document validator does not: it resolves the leaf to a strict
`keyword` and fails when the value is an object. So a document that ingests fine
in a real cluster fails `elastic-package test pipeline`, and the object shape of
a polymorphic field can't be covered by tests.

## Reproduction

Field declaration (GitHub audit, PR elastic/integrations#20306):

```yaml
- name: parameters
type: object
subobjects: false
object_type: keyword
object_type_mapping_type: "*"
- name: parameters.include
type: keyword
```

Test event where `…parameters.include` is an object (the `repository_property`
ruleset target sends this shape):

```json
{ "name": "example-property", "source": "custom", "property_values": ["true"] }
```

`elastic-package test pipeline` fails with:

```
field 'github.ruleset_conditions.parameters.include''s Go type,
map[string]interface {}, does not match the expected field type: keyword
```

The **same document indexes without error** in a live stack — both the string
and object forms are accepted and are queryable via `_field_caps`/ES|QL, with the
object flattened to `…include.name`, `…include.source`, etc.

## Expected vs actual

- **Expected:** when a leaf under `subobjects: false` + `object_type: ` gets an
object value, descend into it and validate its leaves as `` (as ES does).
- **Actual:** the object is rejected as a Go-type mismatch
(`map[string]interface{}` vs `keyword`).

The validator already descends into map values (`validateMapElement`); the gap is
that field resolution collapses `…parameters.include` to its scalar `keyword`
definition and never treats the object value as leaves-under-`object_type`.

## Why it matters

Reviewers reasonably ask for tests covering the object form — but authors can't
provide them: the string form passes, the object form (the exact case the mapping
change fixes) fails the validator. The idiom ships in real packages (`okta`,
`github`), so this blocks legitimate test coverage.

## Related

- Motivating PR: elastic/integrations#20306
- Design proposal: elastic/package-spec#1211

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the validator's validateMapElement entry point and the field-resolution logic described in the issue, then reproduce the failure with elastic-package test pipeline. The fix is done when object values under subobjects: false with object_type are validated through their leaves as the declared type, while the existing scalar form still passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
cli, testing, tooling
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.