4paradigm / 4paradigm/OpenMLDB

abnormal average time when querying different data volume for the same key

Ouverte
#3,871 1 commentaire 0 réactions 2 personnes assignées Réclamée par @aceforeverd Voir sur GitHub
storage-engine
Langage dominant
C++
Étoiles
1.7k
Forks
331
Merge moyen
12 j 12 h
PR mergées (30 j)
1

Description

**Description**
During query performance testing, it was found that querying all data rows for a key incurs the least time cost; querying a subset of data within a specified time range for a key results in a relatively increased time cost; querying data for a specific timestamp for a key results in an even greater increase in time cost.

Detail:
table:import TalkingData train dataset (180+ million rows) into openmldb
query key: ip=88

| key | other condition | rows | average time(us) |
|-------|-------------------------------------------------------|-------|------------------|
| ip=88 | all | 4278 | 9774.375 |
| ip=88 | '2017-11-06 00:00:00' <= ts < "2017-11-07 00:00:00" | 183 | 11570.365 |
| ip=88 | ts='2017-11-06 16:19:38' | 1 | 16504.145 |

more result detail: https://qiok3h8ob4.feishu.cn/docx/YkYfdBZm9oVk0MxLFx9co8lLn1g?from=from_copylink

**Expected Behavior**
querying smaller amounts of data should have shorter time costs, or at least not longer than querying larger amounts of data.

**Steps to Reproduce**

1. deploy openmldb;
2. load data (TalkingData train.csv)into table;
3. find a key with large enough total data volume;
4. execute queries and calculate the average time cost;

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

The issue describes a performance anomaly in OpenMLDB where querying fewer rows for a key takes longer. Start by examining the query execution path for timestamp-based filters versus full key scans. Look at the indexing and data layout for the 'ts' column. Reproduce the issue using the TalkingData dataset and profile the queries to identify bottlenecks.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
sql
Domaine
databases, performance
Type d'issue
Bug
Difficulté
4/5
Temps estimé
3-5 jours
Activité
À l'abandon
Clarté
Plutôt claire
Accessibilité débutants
35/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.