elastic / elastic/integrations
[hackerone] Set lookup index mode on latest report transform for ES|QL LOOKUP JOIN
- Dominant language
- Handlebars
- Stars
- 333
- Forks
- 647
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 209
Description
### Integration Name
HackerOne [packages/hackerone]
### Dataset Name
hackerone.report
### Integration Version
0.1.0
### Agent Version
Latest (aligned with stack)
### OS Version and Architecture
docker image from elastic-package
### User Goal
Configure the HackerOne latest-report transform destination (`logs-hackerone_latest.report`) with **lookup index mode** so it can be used as the right-hand side of ES|QL `LOOKUP JOIN`.
Reference: https://www.elastic.co/docs/reference/query-languages/esql/commands/lookup-join#parameters
### Existing Features
The integration already ships a `latest_report` transform that deduplicates reports into `logs-hackerone_latest.report` (alias → `logs-hackerone_latest.dest_report-v1`). That index works for `FROM` queries and dashboards, but not for `LOOKUP JOIN`, because its index mode is `standard`.
There is no integration setting to enable lookup mode. Manually changing index mode on a Fleet-managed transform destination is not durable across package upgrades.
Current transform destination template (`packages/hackerone/elasticsearch/transform/latest_report/manifest.yml`):
```yaml
start: true
destination_index_template:
mappings:
dynamic: true
dynamic_templates:
- strings_as_keyword:
match_mapping_type: string
mapping:
ignore_above: 1024
type: keyword
date_detection: true
```
### What did you see?
`LOOKUP JOIN` against `logs-hackerone_latest.report` fails because the index is not configured with lookup index mode.
### Anything else?
**Suggested implementation:**
1. Add lookup index mode to `packages/hackerone/elasticsearch/transform/latest_report/manifest.yml`:
```yaml
destination_index_template:
settings:
index:
mode: lookup
mappings:
...
```
2. Bump the destination index suffix in `transform.yml` (`dest_report-v1` → `dest_report-v2`). `index.mode` is immutable after creation; bumping the suffix avoids the upgrade trap seen in Qualys GAV ([#20189](https://github.com/elastic/integrations/issues/20189)).
3. Bump `fleet_transform_version` in `transform.yml` (`0.1.0` → `0.2.0`) to recreate the transform on package upgrade.
4. Document in the integration README that `logs-hackerone_latest.report` supports ES|QL `LOOKUP JOIN` (Stack 9.1+).
5. Release as integration version **`0.2.0`** (minor bump in `manifest.yml` and `changelog.yml`).
**Precedent:** Qualys GAV ([#20067](https://github.com/elastic/integrations/issues/20067)) and CrowdStrike ([packages/crowdstrike/elasticsearch/transform/latest_aidmaster/manifest.yml](https://github.com/elastic/integrations/blob/main/packages/crowdstrike/elasticsearch/transform/latest_aidmaster/manifest.yml)).
cc @kruskall
Contributor guide
Assessment
This issue has not been assessed yet.