fluent / fluent/fluent-bit

Environment Variables are not evaluated before @INCLUDE

Open
#2,020 13 comments 6 reactions 1 assignee Claimed by @edsiper View on GitHub
enhancement
Dominant language
C
Stars
8.1k
Forks
2k
Avg merge
4d 20h
Merged PRs (30d)
71

Description

## Bug Report

**Describe the bug**
When setting the main configuration file to use a different output configuration file based on environment variables, the @ INCLUDE directive does not parse the value of environment variables and instead just directly uses the written line in the configuration file.

**To Reproduce**
`$ export LOG_LEVEL=DEBUG`
main.conf:
```
[SERVICE]
...
[INPUT]
...

@INCLUDE output_match_${LOG_LEVEL}.conf
```
output_match_DEBUG.conf:
```
[OUTPUT]
Name stdout
Match *
```
Terminal output:
`$ /opt/td-agent-bit/bin/td-agent-bit -c /etc/td-agent-bit/main.conf`
```
Fluent Bit v1.3.10
Copyright (C) Treasure Data

[2020/03/10 01:45:10] [Warning] [config] I cannot open /etc/td-agent-bit//output_match_${LOG_LEVEL}.conf file
Error: Configuration file contains errors. Aborting
```
**Expected behavior**
The environment variable should expand to its value and load `output_match_DEBUG.conf`

**Your Environment**
* Version used: v1.3.10 td-agent-bit
* Environment name and version: td-agent-bit via fluent bit apt servers
* Server type and version: Raspberry Pi3
* Operating System and version: Raspbian Debian Buster
* Filters and plugins: no filters or outputs are relevant here

**Additional context**

I have several different matches and outputs I would like to use when various environment variables are set, and I am unable to figure out a way how, though if there is a preferred different way of doing this, I am happy to learn how.

I noticed that there is an "inline config" cli input option for Fluentd, but as far as I can tell there is no equivalent for fluent bit.

Note I am calling this directly using the td-agent-bit binary, and I experienced the same issue when attempting to launch through a service unit file using the `EnvironmentFile` option as well.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.