fluent / fluent/fluent-bit

cannot use default values for environment variables

Open
#5,517 4 comments 12 reactions 0 assignees View on GitHub
enhancement feature-request
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 16h
Merged PRs (30d)
58

Description

## Bug Report

**Describe the bug**
We have a requirement where we have to add certain fields to each logs using env variable and if env variable is not set then set a default value. However if we use `${LOG_TYPE:-docker-collector-logs}` in filter we get error saying [env] variable `${LOG_TYPE:-docker-collector-logs}` is used but not set and we have a core dump

**To Reproduce**
- Steps to reproduce the problem:
1. Create a config as below
```
[SERVICE]
Flush 5
Daemon Off
Log_Level info
Plugins_File plugins.conf
HTTP_Server Off
HTTP_Listen 0.0.0.0
HTTP_Port 2020

[INPUT]
Name cpu
Tag cpu.local
Interval_Sec 1

[FILTER]
Name modify
Match *
Add type ${LOG_TYPE:-docker-collector-logs}

[OUTPUT]
Name stdout
Match *
```
2. set env variable as below
`export LOG_TYPE=test-type`
3. run fluentbit with this config
**Expected behavior**
The environment variable should be used with set value or the default value should be used. Please refer [this](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html). Even tried `${LOG_TYPE:=docker-collector-logs}` but that does not work either

We get the error as below
```
[2022/05/31 10:06:28] [ warn] [env] variable ${LOG_TYPE:=docker-collector-logs} is used but not set
[2022/05/31 10:06:28] [ info] [engine] started (pid=8)
[2022/05/31 10:06:28] [ info] [storage] version=1.1.6, initializing...
[2022/05/31 10:06:28] [ info] [storage] in-memory
[2022/05/31 10:06:28] [ info] [storage] normal synchronization mode, checksum disabled, max_chunks_up=128
[2022/05/31 10:06:28] [ info] [cmetrics] version=0.2.2
[2022/05/31 10:06:28] [error] [filter:modify:modify.0] Invalid operation Add : type in configuration
[2022/05/31 10:06:28] [error] [lib] backend failed
[2022/05/31 10:06:28] [error] Failed initialize filter modify.0
[2022/05/31 10:06:28] [ info] [input] pausing cpu.0
```
Version Used : **1.9.3**

Please note that sometimes the env variables may not be passed and in such cases we would want to use default values (esp for b/w compatibility use cases for logging sidecars when we want to add new fields).

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.