envoyproxy / envoyproxy/envoy

node.extensions does not distinguish between compiled-in and configured bootstrap extensions

Open
#44,333 4 comments 0 reactions 1 assignee Claimed by @mathetake View on GitHub
area/xds enhancement help wanted no stalebot
Dominant language
C++
Stars
28.9k
Forks
5.6k
Avg merge
1d 22h
Merged PRs (30d)
430

Description

*Description*:
Bootstrap extensions that are compiled-in get reported as node.extensions, even when it's not explicitly configured.
However, there are no attributes for the control plane to distinguish between a bootstrap extension that is "compiled-in" vs "compiled-in and configured"

*Repro Steps*:
1. Extension is pulled in via extensions_build_config.bzl
```
"envoy.bootstrap.internal_listener": "//source/extensions/bootstrap/internal_listener:config",
```
2. We want to vend an internal listener on nodes that have envoy.bootstrap.internal_listener configured in bootstrap.json, e.g:
```
"bootstrapExtensions": [
{
"name": "envoy.bootstrap.internal_listener",
"typedConfig": {
"@type": "type.googleapis.com/envoy.extensions.bootstrap.internal_listener.v3.InternalListener"
}
}
],
```
3. However, some nodes may not have internal listener configured despite running the same binary.
4. Control Plane see "envoy.bootstrap.internal_listener" in node.extensions, but has no way of knowing whether the extension is explicitly configured
5. The node that does not explicit register internal listener then sees error:
```
[external/envoy/source/extensions/config_subscription/grpc/delta_subscription_state.cc:282] delta config for type.googleapis.com/envoy.config.listener.v3.Listener rejected: Error adding/updating
listener(s) tunneling_encap_listener: error adding listener named 'tunneling_encap_listener': InternalListener bootstrap extension is mandatory
```

*A few notes*
1. Current workaround is to use node metadata to signal which bootstrap extension is configured
2. Control plane should have a way to distinguish whether an extension is configured or not - either via node or extension state.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.