thingsboard / thingsboard/thingsboard-python-client-sdk

Telemetry upload bug if telemetry list is not ordered by timestamp or if elements of list have different number of key/value pairs

Open
#85 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

verification
Dominant language
Python
Stars
130
Forks
86
PR merge metrics
No merged PRs in 30d

Description

I'm using tb-mqtt-client version 1.10.10

I have a use-case where I'm uploading a list of telemetry readings. Depending on the order of the list, sometimes not all telemetry is being uploaded and sometimes the timestamp with which it is being uploaded is incorrect...

I first though that the list needed to be sorted by the timestamp attribute but then I had cases where even if it was sorted by timestamp it would work fine. It seems to be a combination of the list not being sorted with each element of the list not having the same number of key/value pairs.

I can't figure out or understand this behavior so any help would be appreciated.

This is the code snippet I'm using for testing this problem:

from tb_device_mqtt import TBPublishInfo
from tb_gateway_mqtt import TBGatewayMqttClient

telemetry =[...]

client = TBGatewayMqttClient("my.url.com", port=1883, username="token")
client.connect()

result = client.gw_send_telemetry('test device', telemetry) 

success = result.get() == TBPublishInfo.TB_ERR_SUCCESS
client.disconnect()

print(success)

Following are some examples of the problem where the only thing I'm changing is the order of the telemetry list:

Example 1 - works fine

telemetry =[
    {
        'ts': 1736330010000,
        'values': {
            'test_1': 1,
            'test_1_1': 1,
        }
    },
    {
        'ts': 1736330030000,
        'values': {
            'test_3': 3,
            'test_3_1': 3,
        }
    },
    {
        'ts': 1736330020000,
        'values': {
            'test_2': 2,
            'test_2_1': 2
        }
    },
]

image

Example 2 - does not upload test_2 and test_2_1 telemetry

telemetry =[
    {
        'ts': 1736330010000,
        'values': {
            'test_1': 1,
            'test_1_1': 1
        }
    },
    {
        'ts': 1736330020000,
        'values': {
            'test_2': 2,
            'test_2_1': 2
        }
    },
    {
        'ts': 1736330030000,
        'values': {
            'test_3': 3
        }
    },
]

image

Example 3 - test_1 and test_1_1 are being uploaded with the wrong timestamp. They are being uploaded with the same timestamp as test_3

telemetry =[
    {
        'ts': 1736330010000,
        'values': {
            'test_1': 1,
            'test_1_1': 1,
        }
    },
    {
        'ts': 1736330030000,
        'values': {
            'test_3': 3,
        }
    },
    {
        'ts': 1736330020000,
        'values': {
            'test_2': 2,
            'test_2_1': 2
        }
    },
]

image

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the three telemetry examples through TBGatewayMqttClient.gw_send_telemetry and compare the uploaded results. Trace the telemetry upload path to identify how list order and differing values lengths affect timestamps and entries; done means every value is uploaded with its corresponding timestamp regardless of input order or shape.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.