bytedance / bytedance/videx

[Stats][Bug] Handle converted NULL values before histogram comparisons

Open Beginner friendly
#92 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
149
Forks
27
PR merge metrics
No merged PRs in 30d

Description

### Environment Setup

VIDEX Version: `8cad1b0f07236bc9067afcc36765b35d736e0a14` (current `main` as of 2026-08-22)
Database: standalone statistics path (the repository's MySQL request fixtures)
Deployment Mode: Standalone
OS: Windows 10 Pro N 22H2
Python: 3.9.13
Workload: repository nullable histogram fixtures

### Observed vs Expected Behavior

When a request boundary contains the wire-format string `"NULL"`, `HistogramStats.find_nearest_key_pos()` checks for Python `None` before normalizing the input. `convert_str_by_type()` later converts `"NULL"` to `None`, after which the method immediately compares that value with a histogram bucket.

Actual behavior:

```text
TypeError: '>' not supported between instances of 'NoneType' and 'str'
# or, for an integer histogram:
TypeError: '>' not supported between instances of 'NoneType' and 'int'
```

This currently breaks four repository tests:

```powershell
python -m pytest -q test/videx/test_records_in_range_nullable.py test/videx/test_rec_in_ranges_singleton.py::Test_ask_rec_in_ranges::test_ask_rec_in_ranges_job_072_20a
# 4 failed
```

The shortest failing path is:

```python
value = convert_str_by_type("NULL", data_type, str_in_base4=False)
# value is now None
value > buckets[-1].max_value
# TypeError
```

Expected behavior:

String and Python representations of NULL should follow the same existing NULL-boundary path: the left position is 0 and the right position includes `null_values`, without comparing `None` with a bucket value.

### Impact

Queries whose optimizer trace encodes a nullable range boundary as `"NULL"` fail statistics estimation instead of returning a cardinality. The repository already contains nullable string and integer fixtures that exercise this supported request form.

### Root cause / possible fix

Normalize the request boundary before the NULL branch, while preserving the existing behavior for a literal Python `None` and for histograms with no non-NULL buckets. The existing nullable tests can serve as regression coverage.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at HistogramStats.find_nearest_key_pos() and trace how convert_str_by_type() handles the "NULL" boundary before histogram comparisons. Run the two failing pytest targets: test/videx/test_records_in_range_nullable.py and test/videx/test_rec_in_ranges_singleton.py::Test_ask_rec_in_ranges::test_ask_rec_in_ranges_job_072_20a. Done means nullable string and integer fixtures return cardinality without comparing None to bucket values, while preserving existing Python None and empty non-NULL bucket behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
mysql, python
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.