elastic / elastic/integrations

[hackerone] Set lookup index mode on latest report transform for ES|QL LOOKUP JOIN

Open
#21,114 4 comments 0 reactions 1 assignee Claimed by @clement-fouque View on GitHub
Team:Integration-Experience
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

### Integration Name

HackerOne [packages/hackerone]

### Dataset Name

hackerone.report

### Integration Version

0.1.0

### Agent Version

Latest (aligned with stack)

### OS Version and Architecture

docker image from elastic-package

### User Goal

Configure the HackerOne latest-report transform destination (`logs-hackerone_latest.report`) with **lookup index mode** so it can be used as the right-hand side of ES|QL `LOOKUP JOIN`.

Reference: https://www.elastic.co/docs/reference/query-languages/esql/commands/lookup-join#parameters

### Existing Features

The integration already ships a `latest_report` transform that deduplicates reports into `logs-hackerone_latest.report` (alias → `logs-hackerone_latest.dest_report-v1`). That index works for `FROM` queries and dashboards, but not for `LOOKUP JOIN`, because its index mode is `standard`.

There is no integration setting to enable lookup mode. Manually changing index mode on a Fleet-managed transform destination is not durable across package upgrades.

Current transform destination template (`packages/hackerone/elasticsearch/transform/latest_report/manifest.yml`):

```yaml
start: true
destination_index_template:
mappings:
dynamic: true
dynamic_templates:
- strings_as_keyword:
match_mapping_type: string
mapping:
ignore_above: 1024
type: keyword
date_detection: true
```

### What did you see?

`LOOKUP JOIN` against `logs-hackerone_latest.report` fails because the index is not configured with lookup index mode.

### Anything else?

**Suggested implementation:**

1. Add lookup index mode to `packages/hackerone/elasticsearch/transform/latest_report/manifest.yml`:

```yaml
destination_index_template:
settings:
index:
mode: lookup
mappings:
...
```

2. Bump the destination index suffix in `transform.yml` (`dest_report-v1` → `dest_report-v2`). `index.mode` is immutable after creation; bumping the suffix avoids the upgrade trap seen in Qualys GAV ([#20189](https://github.com/elastic/integrations/issues/20189)).

3. Bump `fleet_transform_version` in `transform.yml` (`0.1.0` → `0.2.0`) to recreate the transform on package upgrade.

4. Document in the integration README that `logs-hackerone_latest.report` supports ES|QL `LOOKUP JOIN` (Stack 9.1+).

5. Release as integration version **`0.2.0`** (minor bump in `manifest.yml` and `changelog.yml`).

**Precedent:** Qualys GAV ([#20067](https://github.com/elastic/integrations/issues/20067)) and CrowdStrike ([packages/crowdstrike/elasticsearch/transform/latest_aidmaster/manifest.yml](https://github.com/elastic/integrations/blob/main/packages/crowdstrike/elasticsearch/transform/latest_aidmaster/manifest.yml)).

cc @kruskall

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.