taosdata / taosdata/TDengine

will stream processing support a periodic mode in the future?

Open
#25,865 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
25.1k
Forks
5k
Avg merge
4d 59m
Merged PRs (30d)
7

Description

目前是有一类需求,简单描述就是希望每隔五分钟计算超级表下两个子表最新行数据的差值。
这个需求来源于在项目中我需要计算电路上下游的电能能耗差,但我的数据会存进两张子表中。
目前的流式计算似乎不能处理两张表及以上的数据(比如两张子表最新的数据列只差、更多子表中第一张表的最新列数据与其他所有表最新列之和的差{t0-(t1+t2)=?}等),如果能将这部分计算也放在数据库中进行,这将显著减少后台从硬件取得数据后需要进行的工作量。

(接下来是幻想时间^ ^)以及我的最理想希望是通过一张超级表记录数据表子表的上下游关系和tag,再依据上下游超级表的子表取出数据超级表的子表来计算我需要的值存入线路超级表,这将完美达成我的需求以使我在设计后台时不需要考虑缓存来自硬件的最新数据(即便我也没有构思出怎样的合法sql可以满足这样通过超级表查询超级表的子表以进行运算)。毕竟数据库就应该是做这个的,而且它天然拥有这种运算所需的一切背景条件 XD

贴一串期望形式的sql,在这个sql中数据超级表将设备所属的线路编号和设备在线路中的序号作为tag:

create stream xd_exam into sline(ts, data_i_need) tags(int lineNo) subtable(concat('line',lineNo)) 
tico 5m 
as 
select now, my_AGGREGATE_udf(m0.data) from
(select last_row(data) from sdata partition by lineNo, seq_in_line) m0
order by seq_in_line partition by lineNo lineNo;

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No implementation files or tests are mentioned. Start by reviewing the stream-processing feature and the documented stream SQL syntax, then compare the proposed periodic, multi-table query with current capabilities. Done would require an agreed design and implementation for periodic processing across related subtables, with validation for the requested calculations.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
databases, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.