splunk / splunk/docker-splunk

Can't force overwrite when using splunk.conf defined files in default.yml

Open
#548 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
549
Forks
277
Avg merge
3d 9h
Merged PRs (30d)
2

Description

I'm trying to get a heavy forwarder going with a Postgres connection, pre-configured to send data to a Spunk Cloud instance.

Eventually I need this to run in Kubernetes, but for now just trying to get it all going with a local postgres container in postgres network.

I got things up and running, but couldn't find a way to force overwrite of an existing conf file which gets installed as part of the DB Connect app, but needs to be updated from the default.

Is there a way to make this work?

The only workaround I found so far was to create the conf files and mount as volume which doesn't seem to get overwritten by the app installer luckily (see commented out bit in Docker Compose file below).

Docker Compose file:

version: "3.6"

# Apps installed from SplunkBase:
# 1. JDBC Add-on for Postgres: https://splunkbase.splunk.com/app/6152
# 2. DB Connect: https://splunkbase.splunk.com/app/2686

services:
  heavyforwarder:
    image: ${SPLUNK_IMAGE:-splunk/splunk:latest}
    hostname: heavyforwarder
    container_name: heavyforwarder
    environment:
      - SPLUNK_START_ARGS=--accept-license
      - SPLUNK_APPS_URL=https://splunkbase.splunk.com/app/6152/release/1.1.0/download,https://splunkbase.splunk.com/app/2686/release/3.9.0/download
      - SPLUNKBASE_USERNAME=myuser
      - SPLUNKBASE_PASSWORD=verysafepassword
      - SPLUNK_PASSWORD=verysafepassword
      - SPLUNK_ROLE=splunk_heavy_forwarder
      - JAVA_VERSION=openjdk:11
      - DEBUG=true
    ports:
      - 8000:8000
    expose:
      - 8000
    networks:
      - ${DOCKER_NETWORK:-shared}
    volumes:
      - ${PWD}/default.yml:/tmp/defaults/default.yml
      # Workaround: have the actual conf files mounted
      # - ${PWD}/config/splunk_app_db_connect:/opt/splunk/etc/apps/splunk_app_db_connect/local

networks:
  shared:
    external:
      name: postgres

default.yml (other config files removed for brevity)

---
splunk:
  conf:
    - key: db_connect_app
      value:
        directory: /opt/splunk/etc/apps/splunk_app_db_connect/local/app.conf
        content:
          install:
            is_configured: 1

Error message:

heavyforwarder    | TASK [splunk_common : Create /opt/splunk/etc/apps/splunk_app_db_connect/local/app.conf directory] ***
heavyforwarder    | fatal: [localhost]: FAILED! => {
heavyforwarder    |     "changed": false,
heavyforwarder    |     "gid": 41812,
heavyforwarder    |     "group": "splunk",
heavyforwarder    |     "mode": "0600",
heavyforwarder    |     "owner": "splunk",
heavyforwarder    |     "path": "/opt/splunk/etc/apps/splunk_app_db_connect/local/app.conf",
heavyforwarder    |     "size": 28,
heavyforwarder    |     "state": "file",
heavyforwarder    |     "uid": 41812
heavyforwarder    | }
heavyforwarder    | 
heavyforwarder    | MSG:
heavyforwarder    | 
heavyforwarder    | /opt/splunk/etc/apps/splunk_app_db_connect/local/app.conf already exists as a file
heavyforwarder    | 

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure using the supplied Docker Compose file and default.yml, focusing on the splunk.conf entry for the existing app.conf path. Trace how these configuration files are processed when the DB Connect app is installed. Done means an explicit overwrite option works for an existing conf file, or the supported limitation is clearly documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, docker-compose, postgres
Domain
databases, devops, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.