It is not possible to set 'smlAutoChildTableNameDelimiter' nor 'smlChildTableName' when trying to insert schemaless OpenTSDBJsonFormatProtocol using the nodeJs connector
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 25.1k
- Forks
- 5k
- Avg merge
- 4d 59m
- Merged PRs (30d)
- 7
Description
Bug Description
When setting smlAutoChildTableNameDelimiter to any value and then trying to insert schemaless data using the InfluxDB Line Protocol via the @tdengine/websocket node package, it results in an error. Using smlChildTableName has no effect, and the table names are the same as when omitting to set this value.
All of the above works when using InfluxDB Line Protocol instead of OpenTSDB JSON protocol but i actually would prefer using the OpenTSDB JSON protocol
I'm using the kubernetes Deployment from https://docs.tdengine.com/operation/deployment/#kubernetes. This has a few problems to start when using as described with tdengine 3.3.3.0 which i could resolve. I had to replace the taoskeeper.toml and set the TAOS_ADAPTER_NODE_EXPORTER_URLS env variable as both had at buildkitsandbox somewhere instead of 127.0.0.1.
To Reproduce
Steps to reproduce the behavior:
- set
smlAutoChildTableNameDelimiterto any value i tried it with '_' - run the example https://docs.tdengine.com/reference/connectors/node/#schemaless-writing removing/commenting out the telnet and influxparts
- The creation of the db works but writing the data fails
- set
smlChildTableNameto 'tname' - run the example https://docs.tdengine.com/reference/connectors/node/#schemaless-writing removing/commenting out the telnet and influxparts and edit the influxdata var to
let jsonData = ["{\"metric\": \"metric_json\",\"tname\":\"some_tname\",\"timestamp\": 1626846400,\"value\": 10.3, \"tags\": {\"groupid\": 2, \"location\": \"California.SanFrancisco\", \"id\": \"d1001\"}}"]; - the generated table does not have the name
some_tnameas defined above but the autogenerated name
Expected Behavior
It should work as described in the documentation https://docs.tdengine.com/reference/schemaless/#main-processing-logic-for-schemaless-writing, which is the case with the InfluxDB Line Protocol
Screenshots
Error when using smlAutoChildTableNameDelimiter
Environment
statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: tdengine
namespace: test
labels:
app: taosd
spec:
replicas: 1
selector:
matchLabels:
app: taosd
template:
metadata:
labels:
app: taosd
spec:
containers:
- name: tdengine
image: tdengine/tdengine:3.3.3.0
ports:
- name: tcp0
containerPort: 6030
protocol: TCP
- name: tcp1
containerPort: 6041
hostPort: 36041
protocol: TCP
envFrom:
- configMapRef:
name: tdengine-config
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: SERVICE_NAME
value: tdengine
- name: STS_NAME
value: tdengine
- name: STS_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
- name: TAOS_SERVER_PORT
value: '6030'
- name: TAOS_FIRST_EP
value: $(STS_NAME)-0.$(SERVICE_NAME).$(STS_NAMESPACE).svc.cluster.local:$(TAOS_SERVER_PORT)
- name: TAOS_FQDN
value: $(POD_NAME).$(SERVICE_NAME).$(STS_NAMESPACE).svc.cluster.local
- name: TAOS_MONITOR_FQDN
value: 'NONE'
- name: TAOS_SML_AUTO_CHILD_TABLE_NAME_DELIMITER
value: '_'
# - name: TAOS_SML_CHILD_TABLE_NAME
# value: 'tname'
- name: TAOS_ADAPTER_NODE_EXPORTER_URLS
value: '["http://localhost:9100"]'
resources: { }
volumeMounts:
- name: tdengine-taosdata
mountPath: /var/lib/taos
- name: tdengine-taoslog
mountPath: /var/log/taos
- name: taos-keeper-toml
mountPath: /etc/taos/taoskeeper.toml
subPath: taoskeeper.toml
readOnly: true
livenessProbe:
exec:
command:
- taos-check
initialDelaySeconds: 15
timeoutSeconds: 1
periodSeconds: 20
successThreshold: 1
failureThreshold: 3
readinessProbe:
exec:
command:
- taos-check
initialDelaySeconds: 5
timeoutSeconds: 5000
periodSeconds: 10
successThreshold: 1
failureThreshold: 3
imagePullPolicy: IfNotPresent
restartPolicy: Always
volumes:
- name: taos-keeper-toml
configMap:
name: taos-keeper-toml-cm
volumeClaimTemplates:
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: tdengine-taosdata
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 5Gi
volumeMode: Filesystem
- kind: PersistentVolumeClaim
apiVersion: v1
metadata:
name: tdengine-taoslog
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
serviceName: tdengine
updateStrategy:
type: RollingUpdate
persistentVolumeClaimRetentionPolicy:
whenDeleted: Retain
whenScaled: Retain
service.yaml
apiVersion: v1
kind: Service
metadata:
name: tdengine
namespace: test
spec:
ports:
- name: tcp0
protocol: TCP
port: 6030
- name: tcp1
protocol: TCP
port: 36041
targetPort: 6041
configmap.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: tdengine-config
namespace: test
data:
CLUSTER: '1'
TAOS_REPLICA: '1'
taos-keeper-toml-cm.yaml
apiVersion: v1
kind: ConfigMap
metadata:
name: taos-keeper-toml-cm
namespace: test
data:
taoskeeper.toml: |
instanceId = 64
# Listen port, default is 6043
port = 6043
# go pool size
gopoolsize = 50000
# interval for metrics
RotationInterval = "15s"
[tdengine]
host = "127.0.0.1"
port = 6041
username = "root"
password = "taosdata"
usessl = false
[metrics]
# metrics prefix in metrics names.
prefix = "taos"
# export some tables that are not super table
tables = []
# database for storing metrics data
[metrics.database]
name = "log"
# database options for db storing metrics data
[metrics.database.options]
vgroups = 1
buffer = 64
keep = 90
cachemodel = "both"
[environment]
# Whether running in cgroup.
incgroup = false
[log]
# The directory where log files are stored.
# path = "/var/log/taos"
level = "info"
# Number of log file rotations before deletion.
rotationCount = 30
# The number of days to retain log files.
keepDays = 30
# The maximum size of a log file before rotation.
rotationSize = "1GB"
# If set to true, log files will be compressed.
compress = false
# Minimum disk space to reserve. Log files will not be written if disk space falls below this limit.
reservedDiskSize = "1GB"
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.