elastic / elastic/integrations

[Logsdb] Strategy to enable logsdb it on stateful & documentation

Open
#12,298 14 comments 1 reaction 0 assignees View on GitHub
Integration:All Team:Fleet
Dominant language
Handlebars
Stars
333
Forks
647
Avg merge
3d 4h
Merged PRs (30d)
209

Description

On serverless, `logsdb` is enabled by default globally.

On stateful, we need to define and document how to:
- Enable `logsdb` on a single integration (as long as the data stream is `logs`)
- Enable `logsdb` on all `logs-*` integrations

Considerations:
- Applicable only to 8.17+ as Logsdb became GA
- We need to make sure `logsdb` is safe to be enabled on all integrations
- If this assumption is not correct, then we need to find a way to mark integrations as "logsdb-compatible"

## Strategies

### Possible strategies (manual)

Those strategies should work (tested in 8.17), but they're not yet documented

A) To enable `logsdb` across all `logs-*` data streams.
- set `cluster.logsdb.enabled: true` ~and restart the master node (?)~
- on next rollover, ANY data stream matching `logs-*-*` will start using Logsdb (it is injected live by ES code https://github.com/elastic/elasticsearch/pull/113505)

B) To enable `logsdb` across all `logs-*` data streams which follow the Fleet Index Template structure
- create/update the `logs@custom` component template and add `settings.index.mode` to `logsdb`
- rollover manually or wait next rollover
Risks:
- If we change `settings.index.mode` in the future, users might have the burden or take all the `@custom` component templates and update them

C) To enable `logsdb` on a specific `logs-*` data stream belonging to an integration.
- create/update the `logs-@custom` component template and add `settings.index.mode` to `logsdb`
- rollover manually or wait next rollover
Risks:
- If we change `settings.index.mode` in the future, users might have the burden or take all the `@custom` component templates and update them
- Users might attempt to use this on non-logs data streams...

### Possible strategies (via UI)

Offer some "opt-in" button to automate the processes above via Stack Management.

### Related

- https://github.com/elastic/elasticsearch/issues/106489

### Relevant SMEs

- @jsoriano
- @felixbarny
- @ruflin
- @martijnvg
- feel free to add/edit

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.