taosdata / taosdata/TDengine

Authentication failure on one pod after cluster upgrade from 3.3.6.9 to 3.3.7.5

Open
#33,566 6 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
25.1k
Forks
5k
Avg merge
4d 59m
Merged PRs (30d)
7

Description

Bug Description

After upgrading our TDengine cluster from version 3.3.6.9 to 3.3.7.5, the cluster runs normally for a period of time, but then one of the pods starts experiencing authentication failures when attempting to send commands to the mnode using the default password. The command should use the actual database password, not the default.

To Reproduce
Steps to reproduce the behavior:

  1. Upgrade TDengine cluster from version 3.3.6.9 to 3.3.7.5
  2. Cluster operates normally for a few hours
  3. One pod begins failing authentication when connecting to mnode using default credentials

Expected Behavior
All pods should maintain consistent authentication after the upgrade, given that no password changes were made during or after the upgrade process.

Actual Behaviour
One pod fails authentication when trying to connect to mnode
The password was not explicitly changed during or after the upgrade
Eventually, only the mnode pod continue working normally
Error appears spontaneously after the cluster has been running for some time

Screenshots
Pod Logs

Image

Environment (please complete the following information):

TDengine Version (before upgrade): 3.3.6.9
TDengine Version (after upgrade): 3.3.7.5
Deployment: Kubernetes cluster
Platform: Linux

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the attached pod logs and reproduce the upgrade from TDengine 3.3.6.9 to 3.3.7.5 in a Kubernetes cluster. Trace the pod-to-mnode authentication command and compare the credential it uses with the configured database password. Done means all pods retain consistent authentication after several hours without password changes.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, linux
Domain
authentication, databases, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.