elastic / elastic/ecs-dotnet

[BUG] Synthetic Source Compatibility Error in Elastic.Serilog.Sinks 9.x

Open
#532 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
HTML
Stars
137
Forks
70
PR merge metrics
No merged PRs in 30d

Description

**ECS integration/library project(s) (e.g. Elastic.CommonSchema.Serilog)**: Elastic.Serilog.Sinks (Elastic.CommonSchema.Elasticsearch)

**ECS schema version**: 9.0.0

**ECS .NET assembly version**: 9.0.0

**Elasticsearch version**: 8.13.2

**.NET framework / OS**: .NET Core 9 / Windows Server 2020

**Description of the problem, including expected versus actual behavior**:
After upgrading to Elastic.Serilog.Sinks 9.x, the library attempts to create component templates with the `synthetic_source_keep` parameter, which is not supported in some Elasticsearch 8.13.2 configurations, despite the version number suggesting compatibility.

**Error Message**:
```
Unhandled exception. System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.Exception: Failure to create component template `ecs_9.0.0_base` for logs-srv-*: Invalid Elasticsearch response built from a unsuccessful (400) low level call on PUT: /_component_template/ecs_9.0.0_base
Exception: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""

# Audit trail of this API call:
- [1] BadResponse: Node: http://localhost:9200/ Took: 00:00:00.0190869
# OriginalException: Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
# Request:
{
"_meta": {
"documentation": "https://www.elastic.co/guide/en/ecs/current/ecs-base.html",
"ecs_version": "9.0.0"
},
"template": {
"mappings": {
"properties": {
"@timestamp": {
"type": "date"
},
"labels": {
"type": "object"
},
"message": {
"type": "match_only_text"
},
"tags": {
"ignore_above": 1024,
"synthetic_source_keep": "none",
"type": "keyword"
}
}
}
}
}

# Response:
{"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}],"type":"mapper_parsing_exception","reason":"Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]","caused_by":{"type":"mapper_parsing_exception","reason":"unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]"}},"status":400}

---> Elastic.Transport.TransportException: Request failed to execute. Call: Status code 400 from: PUT /_component_template/ecs_9.0.0_base. ServerError: Type: mapper_parsing_exception Reason: "Failed to parse mapping: unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]" CausedBy: "Type: mapper_parsing_exception Reason: "unknown parameter [synthetic_source_keep] on mapper [tags] of type [keyword]""
--- End of inner exception stack trace ---
at Elastic.Ingest.Elasticsearch.ElasticsearchChannelBase`2.PutComponentTemplate(BootstrapMethod bootstrapMethod, String name, String body)
at Elastic.Ingest.Elasticsearch.CommonSchema.EcsDataStreamChannel`1.BootstrapElasticsearch(BootstrapMethod bootstrapMethod, String ilmPolicy)
at Elastic.Serilog.Sinks.ElasticsearchSink`1..ctor(ElasticsearchSinkOptions`1 options)
at Elastic.Serilog.Sinks.ElasticsearchSink..ctor(ElasticsearchSinkOptions options)
at Elastic.Serilog.Sinks.ConfigSinkExtensions.Elasticsearch(LoggerSinkConfiguration loggerConfiguration, BootstrapMethod bootstrapMethod, ICollection`1 nodes, Boolean useSniffing, String dataStream, String ilmPolicy, String apiKey, String username, String password, Nullable`1 includeHost, Nullable`1 includeActivity, Nullable`1 includeProcess, Nullable`1 includeUser, ICollection`1 filterProperties, Nullable`1 maxRetries, Nullable`1 maxConcurrency, Nullable`1 maxInflight, Nullable`1 maxExportSize, Nullable`1 maxLifeTime, Nullable`1 fullMode, Uri proxy, String proxyUsername, String proxyPassword, String fingerprint, Boolean debugMode, LoggingLevelSwitch levelSwitch, LogEventLevel restrictedToMinimumLevel)
at System.RuntimeMethodHandle.InvokeMethod(Object target, Void** arguments, Signature sig, Boolean isConstructor)
at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
--- End of inner exception stack trace ---
at System.Reflection.MethodBaseInvoker.InvokeWithManyArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
at Serilog.Settings.Configuration.ConfigurationReader.CallConfigurationMethods(ILookup`2 methods, IReadOnlyCollection`1 configurationMethods, Object receiver)
at Serilog.Settings.Configuration.ConfigurationReader.ApplySinks(LoggerConfiguration loggerConfiguration)
at Serilog.Settings.Configuration.ConfigurationReader.Configure(LoggerConfiguration loggerConfiguration)
at Serilog.Configuration.LoggerSettingsConfiguration.Settings(ILoggerSettings settings)
at Serilog.ConfigurationLoggerConfigurationExtensions.Configuration(LoggerSettingsConfiguration settingConfiguration, IConfiguration configuration, ConfigurationReaderOptions readerOptions)
at Program.$(String[] args)
at Program.(String[] args)
```

**Steps to reproduce**:
1. Have an Elasticsearch 8.13.2 cluster (free version)
2. Install Elastic.Serilog.Sinks 9.0.0

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.