elastic / elastic/ecs-dotnet

[Question] Pipelines execution in Elastic.Serilog.Sinks

Open
#517 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
137
Forks
70
PR merge metrics
No merged PRs in 30d

Description

Hello,
During migration from `Serilog.Sinks.Elasticsearch` to `Elastic.Serilog.Sinks`, I encountered missing functionality for configuring the ingest pipeline in `ElasticsearchSinkOptions`. The pipeline is used to set up preprocessors that select the index based on log field values and to drop some redundant logs. Is there any other way to specify the pipeline? I couldn’t find any similar issues except [this unanswered one](https://discuss.elastic.co/t/pipelines-execution-in-elastic-serilog-sinks/374056). If there is a way to specify the pipeline, please share it. It would be helpful to have the ability to specify which pipeline should be used.

Here is `Serilog.Sinks.Elasticsearch` example:
```csharp
var serilogSinksElasticsearchOptions = new ElasticsearchSinkOptions(new Uri("{elasticUrl}"))
{
PipelineName = "my-pipeline",
CustomFormatter = new ElasticsearchJsonFormatter(),
BatchAction = ElasticOpType.Create,
ConnectionTimeout = TimeSpan.FromSeconds(30),
ModifyConnectionSettings = connection => connection
.BasicAuthentication("username", "password")
.RequestTimeout(TimeSpan.FromSeconds(90))
.MaximumRetries(3),
};
```
And corresponding `Elastic.Serilog.Sinks`:
```csharp
loggerConfig.WriteTo.Elasticsearch([new Uri("{elsaticUrl}")],
options =>
{
// options.PipelineName = "my-pipeline"; -- unfortunately no such option
options.BootstrapMethod = BootstrapMethod.None;
options.DataStream = new DataStreamName("my-data-stream");
options.TextFormatting = new EcsTextFormatterConfiguration();
options.ConfigureChannel = channelOptions =>
{
channelOptions.BufferOptions = new BufferOptions();
};
options.IlmPolicy = DataStreamName;
},
transport =>
{
transport.Authentication(new BasicAuthentication("username", "password"));
transport.RequestTimeout(TimeSpan.FromSeconds(90));
transport.MaximumRetries(3);
});
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.