microsoft / microsoft/azure-container-apps

[KEDA][Kafka] DNS resolution not working with a custom DNS server

Open
#970 4 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Bicep
Stars
445
Forks
54
Avg merge
5h 14m
Merged PRs (30d)
3

Description

This issue is a: (mark with an x)
  • [ x] bug report -> please search issues before submitting
  • documentation issue or request
  • regression (a behavior that used to work and stopped in a new release)
Issue description

We use custom kafka KEDA rule on a pod, but the dns resolution doesn't work as expected.
Our Kafka cluster is linked to a private endpoint.
So the dns resolution inside a pod is
xxxx.westeurope.azure.confluent.cloud => X.240.Y.Z
and from KEDA
xxxx.westeurope.azure.confluent.cloud =>10.1.0.4

our scaling rule :

"scale": {
    "minReplicas": 1,
    "maxReplicas": 6,
    "rules": [
        {
            "name": "kafka-rule",
            "custom": {
                "type": "kafka",
                "metadata": {
                    "bootstrapServers": "xxxx.westeurope.azure.confluent.cloud:9092",
                    "consumerGroup": "myconsumergroup",
                    "lagThreshold": "20",
                    "offsetResetPolicy": "earliest",
                    "sasl": "plaintext",
                    "tls": "enable",
                    "topic": "mytopic"
                },
                "auth": [
                    {
                        "secretRef": "kafka-keda-username",
                        "triggerParameter": "username"
                    },
                    {
                        "secretRef": "kafka-keda-password",
                        "triggerParameter": "password"
                    }
                ]
            }
        }
    ]
},

note that the kafka cluster url has a public resolution => 10.1.0.4, but in our network is X.240.Y.Z.

Steps to reproduce
  1. use confluent KAfka cluster with private connectivity
  2. create Azure VNET and set DNS servers to custom
  3. Create Container App Environment in subnet of previous vnet (Workload profile is used)
  4. Create a Container App that consume Kafka (Dapr is not used)
  5. Setup scaling rule as above

Expected behavior [What you expected to happen.]

KEDA should resolve url 'xxxx.westeurope.azure.confluent.cloud' to X.240.Y.Z

Actual behavior [What actually happened.]

KEDA resolve url 'xxxx.westeurope.azure.confluent.cloud' to 10.1.0.4, so scaling doesn't works

Screenshots
If applicable, add screenshots to help explain your problem.
extacted from system logs :
image

VNET DNS configuration :
image

workload profile configuration :
image

Additional context

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Begin with the provided reproduction steps for the Container Apps environment, custom VNET DNS, and Kafka scaling rule. Inspect the linked system logs and compare DNS resolution inside the consuming app with resolution from KEDA. Done means the Kafka hostname resolves to the private address X.240.Y.Z and scaling works.

Written by the indexing model from the issue text.

Assessment

Tech stack
azure, kafka
Domain
cloud, networking
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.