taosdata / taosdata/TDengine

error upgrading tdengine from 3.3.6.0 to 3.3.6.3 in kubernetes

Open
#31,097 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
C
Stars
25.1k
Forks
5k
Avg merge
4d 59m
Merged PRs (30d)
7

Description

k8s版本:
Client Version: v1.31.0
Kustomize Version: v5.4.2
Server Version: v1.31.7-eks-bcf3d70
helm chart :3.0.2
实践记录:
1 先升级后写入数据(没有问题)
在跑3.3.6.0的镜像时,会出现正常情况pod内进程如下

Image

更换成3.3.6.3之后:

Image

将镜像版本由td-engine由3.3.6.0升级到3.3.6.3(之前没有写入数据),就不会报错: 日志如下:
pod也是健康的:

Image

执行如下命令写入数据也没问题
taosBenchmark -d db -t 100 -n 1000 -T 4 -I stmt -y
2 先写入数据后升级(出现问题)
先准备个tdengine3.3.6.0 3节点集群

Image

Image

然后写入数据:
taosBenchmark -d db -t 100 -n 1000 -T 4 -I stmt -y

再改镜像:

Image

Image

pod滚动更新,接下来会看到日志报错:
pod-2:

Image

进程都正常:

Image

pod-1:

Image

taoskeeper没起来:

Image

pod-0:

Image

taoskeeper没起来:

Image

3 直接将镜像改成3.3.6.3 (出现问题)

Image
将镜像直接变成3.6.6.3,镜像直接起不来

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the upgrade sequence with Kubernetes v1.31, Helm chart 3.0.2, and TDengine images 3.3.6.0 and 3.3.6.3. Start with the taoskeeper and pod logs while running taosBenchmark, comparing upgrades before and after data is written. Done means a three-node cluster with existing data upgrades successfully and all pods remain healthy.

Written by the indexing model from the issue text.

Assessment

Tech stack
helm, kubernetes
Domain
databases, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.