splunk / splunk/splunk-sdk-python

connect() fails on FIPS-enabled Splunk 10.4.x: unguarded ctx.set_groups() raises ssl.SSLError (_ssl.c:4981) before any connection is attempted

Open Beginner friendly
#835 1 comment 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
743
Forks
387
Avg merge
42m
Merged PRs (30d)
4

Description

Summary

splunklib/binding.py connect() (the verify=False path — the splunklib default) configures an explicit TLS key-exchange group list that includes ML-KEM hybrid groups:

if hasattr(ctx, "set_groups"):
    ctx.set_groups(  # pyright: ignore[reportUnknownMemberType, reportAttributeAccessIssue]
        "X25519MLKEM768:SecP256r1MLKEM768:SecP384r1MLKEM1024:"
        + "MLKEM512:MLKEM768:MLKEM1024:"
        + "X25519:secp256r1:X448:secp384r1:secp521r1:ffdhe2048:ffdhe3072"
    )

(binding.py lines 1744-1758 in splunk-sdk 3.0.0; identical on develop today.)

As the adjacent comment notes, SSLContext.set_groups is a Python 3.15 API that Splunk backports into its patched 3.9/3.13 runtimes — so on Splunk builds that carry the backport, the hasattr guard passes. But there is no exception handling: on a FIPS-enabled Splunk deployment, the OpenSSL FIPS provider has no validated ML-KEM, SSL_CTX_set1_groups_list rejects the group list, and every client.connect() (and any app-level code using splunklib against splunkd) dies before a single packet is sent:

File ".../lib/splunklib/binding.py", line 1754, in connect
    ctx.set_groups(
ssl.SSLError: unknown error (_ssl.c:4981)

This breaks any Splunk app that bundles splunk-sdk 3.0.0 on a FIPS-enabled Splunk 10.4.x search head — in our case, all REST handlers of a widely deployed app became inoperable after the customer-side platform upgrade, surfacing as HTTP 500 on every request.

Environment / verified matrix

Splunk Runtime OpenSSL hasattr(ctx, "set_groups") ctx.set_groups(<SDK list>)
10.2.4 (1526e5e5df42), FIPS bundled Python 3.9.25 and 3.13.11 3.0.19 False n/a — SDK skips the call, everything works
10.4.2 (33c3bf42cd73), FIPS bundled Python 3.13.11 (what python3 resolves to) 3.5.7 True SSLError(0, 'unknown error (_ssl.c:4981)')

Per-group-name bisection on 10.4.2 FIPS (splunk cmd python3, ssl._create_unverified_context()):

  • FAIL: X25519MLKEM768, SecP256r1MLKEM768, SecP384r1MLKEM1024, MLKEM512, MLKEM768, MLKEM1024 — each raises SSLError(0, 'unknown error (_ssl.c:4981)')
  • OK: X25519, secp256r1, X448, secp384r1, secp521r1, ffdhe2048, ffdhe3072

So the failure needs no app code at all to reproduce — the FIPS provider simply rejects every ML-KEM group name in the hardcoded list.

Minimal reproduction (on a FIPS-enabled Splunk 10.4.2)

$SPLUNK_HOME/bin/splunk cmd python3 - <<'EOF'
import ssl
ctx = ssl._create_unverified_context()
print("has set_groups:", hasattr(ctx, "set_groups"))   # True on 10.4.2
ctx.set_groups("X25519MLKEM768:SecP256r1MLKEM768:SecP384r1MLKEM1024:"
               "MLKEM512:MLKEM768:MLKEM1024:"
               "X25519:secp256r1:X448:secp384r1:secp521r1:ffdhe2048:ffdhe3072")
EOF
# → ssl.SSLError: unknown error (_ssl.c:4981)

Any splunklib.client.connect(...) on such a host then fails identically inside binding.py connect().

Suggested fix

Wrap the call so a rejected group list degrades to the context's default groups — the exact behaviour every pre-backport Splunk runs today:

if hasattr(ctx, "set_groups"):
    try:
        ctx.set_groups(
            "X25519MLKEM768:SecP256r1MLKEM768:SecP384r1MLKEM1024:"
            + "MLKEM512:MLKEM768:MLKEM1024:"
            + "X25519:secp256r1:X448:secp384r1:secp521r1:ffdhe2048:ffdhe3072"
        )
    except ssl.SSLError:
        # FIPS providers (and OpenSSL builds without ML-KEM) reject the
        # ML-KEM group names — fall back to the default groups rather
        # than failing every connection.
        pass

Optionally retry with the classical-only tail (X25519:secp256r1:X448:secp384r1:secp521r1:ffdhe2048:ffdhe3072 — all accepted by the FIPS provider in our bisection) before falling back entirely, but a plain fallback already restores full functionality. Happy to submit a PR for either variant.

Workaround we ship meanwhile

Our app's build pipeline patches the bundled binding.py to add exactly the try/except above. Validated: the patched build fully restores functionality on the FIPS-enabled Splunk 10.4.2 reproduction host (same host, same runtime — all REST handlers operational again), confirming the fallback-to-default-groups approach is sufficient.


AI disclosure

This issue was investigated, reproduced, and written with the assistance of an AI coding agent (Claude Code by Anthropic), operated and reviewed by Guilhem Marchand (TrackMe Limited). All reproduction outputs quoted above were produced on real Splunk hosts and human-verified before filing.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in splunklib/binding.py at connect(), lines 1744-1758, and run the minimal set_groups reproduction against a FIPS-enabled Splunk 10.4.2 runtime. Guard the group configuration failure so connections fall back to the context defaults, then verify that client.connect() and the affected REST handlers proceed without the SSL error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.