Environment variables are not resolved when using templates

It seems that environment variables are not resolved the same way in templates as they are in regular configuration files. There is a workaround, so it's not a major issue, but I assume this could be a challenge for many newcomers, as it was for me.

Given this template:

type: input
name: kafka
fields:
  - name: topic
    type: string

mapping: |
  #!blobl
  root = {
    "kafka_franz": {
      "seed_brokers": [ "${KAFKA_BROKERS}" ],
      "topics": [ this.topic ],
      "sasl": ...
    }
}

Here's the error I encounter when using templates:

WARN unable to open connection to broker           @service=redpanda-connect addr="${KAFKA_BROKERS}:9092" broker=seed_0 err="dial tcp: lookup ${KAFKA_BROKERS}: no such host" label="" path=root.input
(edited)

It's simple to replace seed_brokers with env("KAFKA_BROKERS"), if you know you need to, but this is a bit of a gotcha.

Moreover, it seems you can't set defaults using environment variables. Both default: "${SOME_VARIABLE}" and default: ${! env("SOME_VARIABLE") } appear to evaluate to null, resulting in the error: expected object value, got !!null.

Start by reproducing the template YAML example with ${KAFKA_BROKERS} and compare it with the equivalent regular configuration behavior. Investigate the template handling for broker values and defaults using env("KAFKA_BROKERS"); done means environment variables resolve consistently in templates, including defaults, without producing null values.