4paradigm / 4paradigm/OpenMLDB
After inserting data with precompiled SQL, when queried, certain fields have additional null characters ('\x00') appended at the end
- 主要语言
- C++
- 星标
- 1.7k
- 派生
- 331
- 平均合并
- 12 天 12 小时
- 30 天内合并 PR
- 1
描述
**Bug Description**
Service Version: Code from the main branch after version 0.8.4, compiled and deployed on macOS 13.5.1.
Client Version: The current project branch python-sdk, accessing the database using the dbapi method.
After inserting data with precompiled SQL, when queried, certain fields have additional null characters ('\x00') appended at the end, as shown in the fourth column:
`
[(1001, '2022-05-01', 'province1', 'city1\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 1, 1590738991000),
(1002, '2022-05-02', 'province2', 'city2\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 2, 1590738992000),
(1003, '2022-05-03', 'province3', 'city3\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 3, 1590738993000),
(1004, '2022-05-04', 'province0', 'city4\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00', 4, 1590738994000)]
`
**Expected Behavior**
The field contents should not have additional null characters appended and should match the inserted content.
**Relation Case**
**Steps to Reproduce**
1. Use the python-sdk code from the main branch after version 0.8.4.
2. Run the test cases below to reproduce the corresponding error.
```
import sys
import os
from openmldb.dbapi import connect
import pytest
from .case_conf import OpenMLDB_ZK_CLUSTER, OpenMLDB_ZK_PATH
class TestOpenmldbDBAPI:
cursor_without_db = None
cursor = None
db_name = "dbapi_test"
@classmethod
def setup_class(cls):
# connect without db to create it
db = connect(zk=OpenMLDB_ZK_CLUSTER, zkPath=OpenMLDB_ZK_PATH)
cls.cursor_without_db = db.cursor()
cls.cursor_without_db.execute(
f"create database if not exists {cls.db_name};")
# then we can use the db to connect
db = connect(database=cls.db_name,
zk=OpenMLDB_ZK_CLUSTER,
zkPath=OpenMLDB_ZK_PATH)
cls.cursor = db.cursor()
cls.cursor.execute("create database if not exists {};".format(cls.db_name))
assert cls.db_name in cls.cursor.get_databases()
def recreate_table(self, table, schema):
if table in self.cursor.get_tables(self.db_name):
self.cursor.execute("drop table {}".format(table))
assert table not in self.cursor.get_tables(self.db_name)
self.cursor.execute("create table {}({}) OPTIONS(partitionnum=1);".format(table, schema))
assert table in self.cursor.get_tables(self.db_name)
def test_more_parameterized_query(self):
table = "test_param_query"
schema_str = "col1 bigint, col2 date, col3 string, col4 string, col5 int, col6 timestamp, " \
"index(key=col3, ts=col1), index(key=col3, ts=col6)"
self.recreate_table(table, schema_str)
test_rows = [(1000 + i, '2022-05-0' + str(i), 'province' + str(i % 4),
'city' + str(i), i, (1590738990 + i) * 1000)
for i in range(1, 5)]
self.cursor.executemany('insert into {} values (?, ?, ?, ?, ?, ?);'.format(table), test_rows)
result = self.cursor.execute("select * from {};".format(table)).fetchall()
result = sorted(list(result), key=lambda x: x[0])
assert result == test_rows
if __name__ == "__main__":
sys.exit(pytest.main(["-vv", os.path.abspath(__file__)]))
```
贡献指南
调研方向
该 issue 包含一个失败的 Python 测试用例,该用例通过 OpenMLDB Python SDK 插入并查询数据。首先运行提供的测试,以复现 null 字符填充问题。检查 C++ 数据库引擎在预编译 SQL 中对 string 字段的处理,尤其是内存分配和填充逻辑。检查 Python SDK 的 dbapi 层是否存在 string 编码/解码问题。目标是确保返回的 string 不包含额外的 null 字符。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python, sql
- 领域
- backend-api-design, databases, machine-learning
- Issue 类型
- 缺陷
- 难度
- 3/5
- 预计耗时
- 1-2 天
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 45/100