[Question] Pipelines execution in Elastic.Serilog.Sinks
- Dominant language
- HTML
- Stars
- 137
- Forks
- 70
- PR merge metrics
- No merged PRs in 30d
Description
Hello,
During migration from `Serilog.Sinks.Elasticsearch` to `Elastic.Serilog.Sinks`, I encountered missing functionality for configuring the ingest pipeline in `ElasticsearchSinkOptions`. The pipeline is used to set up preprocessors that select the index based on log field values and to drop some redundant logs. Is there any other way to specify the pipeline? I couldn’t find any similar issues except [this unanswered one](https://discuss.elastic.co/t/pipelines-execution-in-elastic-serilog-sinks/374056). If there is a way to specify the pipeline, please share it. It would be helpful to have the ability to specify which pipeline should be used.
Here is `Serilog.Sinks.Elasticsearch` example:
```csharp
var serilogSinksElasticsearchOptions = new ElasticsearchSinkOptions(new Uri("{elasticUrl}"))
{
PipelineName = "my-pipeline",
CustomFormatter = new ElasticsearchJsonFormatter(),
BatchAction = ElasticOpType.Create,
ConnectionTimeout = TimeSpan.FromSeconds(30),
ModifyConnectionSettings = connection => connection
.BasicAuthentication("username", "password")
.RequestTimeout(TimeSpan.FromSeconds(90))
.MaximumRetries(3),
};
```
And corresponding `Elastic.Serilog.Sinks`:
```csharp
loggerConfig.WriteTo.Elasticsearch([new Uri("{elsaticUrl}")],
options =>
{
// options.PipelineName = "my-pipeline"; -- unfortunately no such option
options.BootstrapMethod = BootstrapMethod.None;
options.DataStream = new DataStreamName("my-data-stream");
options.TextFormatting = new EcsTextFormatterConfiguration();
options.ConfigureChannel = channelOptions =>
{
channelOptions.BufferOptions = new BufferOptions();
};
options.IlmPolicy = DataStreamName;
},
transport =>
{
transport.Authentication(new BasicAuthentication("username", "password"));
transport.RequestTimeout(TimeSpan.FromSeconds(90));
transport.MaximumRetries(3);
});
```
Contributor guide
Assessment
This issue has not been assessed yet.