microsoft / microsoft/azure-container-apps
[KEDA][Kafka] DNS resolution not working with a custom DNS server
Nobody has claimed this yet.
- Dominant language
- Bicep
- Stars
- 445
- Forks
- 54
- Avg merge
- 5h 14m
- Merged PRs (30d)
- 3
Description
This issue is a: (mark with an x)
- [ x] bug report -> please search issues before submitting
- documentation issue or request
- regression (a behavior that used to work and stopped in a new release)
Issue description
We use custom kafka KEDA rule on a pod, but the dns resolution doesn't work as expected.
Our Kafka cluster is linked to a private endpoint.
So the dns resolution inside a pod is
xxxx.westeurope.azure.confluent.cloud => X.240.Y.Z
and from KEDA
xxxx.westeurope.azure.confluent.cloud =>10.1.0.4
our scaling rule :
"scale": {
"minReplicas": 1,
"maxReplicas": 6,
"rules": [
{
"name": "kafka-rule",
"custom": {
"type": "kafka",
"metadata": {
"bootstrapServers": "xxxx.westeurope.azure.confluent.cloud:9092",
"consumerGroup": "myconsumergroup",
"lagThreshold": "20",
"offsetResetPolicy": "earliest",
"sasl": "plaintext",
"tls": "enable",
"topic": "mytopic"
},
"auth": [
{
"secretRef": "kafka-keda-username",
"triggerParameter": "username"
},
{
"secretRef": "kafka-keda-password",
"triggerParameter": "password"
}
]
}
}
]
},
note that the kafka cluster url has a public resolution => 10.1.0.4, but in our network is X.240.Y.Z.
Steps to reproduce
- use confluent KAfka cluster with private connectivity
- create Azure VNET and set DNS servers to custom
- Create Container App Environment in subnet of previous vnet (Workload profile is used)
- Create a Container App that consume Kafka (Dapr is not used)
- Setup scaling rule as above
Expected behavior [What you expected to happen.]
KEDA should resolve url 'xxxx.westeurope.azure.confluent.cloud' to X.240.Y.Z
Actual behavior [What actually happened.]
KEDA resolve url 'xxxx.westeurope.azure.confluent.cloud' to 10.1.0.4, so scaling doesn't works
Screenshots
If applicable, add screenshots to help explain your problem.
extacted from system logs :
VNET DNS configuration :
workload profile configuration :
Additional context
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Begin with the provided reproduction steps for the Container Apps environment, custom VNET DNS, and Kafka scaling rule. Inspect the linked system logs and compare DNS resolution inside the consuming app with resolution from KEDA. Done means the Kafka hostname resolves to the private address X.240.Y.Z and scaling works.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- azure, kafka
- Domain
- cloud, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100