elastic / elastic/beats

Regression in Logstash FIPS tests: filebeat panics during TLS 1.3 handshake with OpenSSL 3.6.2

Open
#50,175 6 comments 0 reactions 0 assignees View on GitHub
needs_team
Dominant language
Go
Stars
12.7k
Forks
5k
Avg merge
2d 15m
Merged PRs (30d)
385

Description

## Summary

Filebeat FIPS 8.19.15 crashes with a goroutine panic on every TLS 1.3 connection. This breaks the Filebeat → Logstash → ES pipeline in the observability SRE acceptance tests.

```
panic: failed to add parameter data: OSSL_PARAM_BLD_push_octet_string
openssl error(s):
error:078C0102:common libcrypto routines::passed a null parameter

goroutine 88 [running]:
crypto/tls/internal/tls13.ExpandLabel[...]
crypto/tls/internal/tls13/tls13.go:40
crypto/tls.(*cipherSuiteTLS13).trafficKey(...)
crypto/tls/key_schedule.go:28
crypto/tls.(*clientHandshakeStateTLS13).establishHandshakeKeys(...)
crypto/tls/handshake_client_tls13.go:518
```

## Root Cause

OpenSSL 3.6.2 added a NULL check to `OSSL_PARAM_BLD_push_octet_string` that rejects nil buffers. Go's TLS 1.3 `ExpandLabel` passes a nil context through `golang-fips/openssl`, which now panics instead of being treated as empty.

## Existing Upstream Issues

- **OpenSSL regression**: https://github.com/openssl/openssl/issues/30728 — fix merged in https://github.com/openssl/openssl/pull/30730
- **golang-fips/openssl workaround**: https://github.com/golang-fips/openssl/pull/417

## Reproduction

The Logstash observability SRE acceptance tests expose this via the [filebeat-to-ls-to-es.conf](https://github.com/elastic/logstash/blob/8.19/x-pack/distributions/internal/observabilitySRE/qa/acceptance/docker/logstash/pipeline/filebeat-to-ls-to-es.conf) pipeline, which configures a Beats input with TLS:

```bash
export OBSERVABILITY_SRE_IMAGE_VERSION="8.19.15-SNAPSHOT"
export ELASTICSEARCH_IMAGE_VERSION="8.19.15-SNAPSHOT"
export FILEBEAT_IMAGE_VERSION="8.19.15-SNAPSHOT"
./gradlew observabilitySREacceptanceTests --stacktrace
```

3 of 4 tests pass. The "Filebeat through LS to ES in a FIPS compliant configuration" test fails because Filebeat panics before sending any data.

## Environment

| Component | Version |
|-----------|---------|
| Filebeat | 8.19.15-SNAPSHOT FIPS (`elasticsearch-cloud-ess-fips`) |
| Go | 1.25.8 |
| OpenSSL | 3.6.2 |
| Logstash | 8.19.15-SNAPSHOT observability-SRE image |

Contributor guide

Open the contributing guide

Research direction

Start with the observability SRE acceptance command and the pipeline in x-pack/distributions/internal/observabilitySRE/qa/acceptance/docker/logstash/pipeline/filebeat-to-ls-to-es.conf. Trace the Filebeat FIPS TLS 1.3 handshake and compare the reported nil-buffer behavior with the linked golang-fips/openssl workaround. Done means the FIPS acceptance test completes without a panic and the Filebeat-to-Logstash-to-Elasticsearch pipeline sends data successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
networking, observability-sre, security
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.