astronomy-commons / astronomy-commons/lsdb-server

lsdb-server cannot filter based on string columns

Open
#11 1 comment 0 reactions 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
2
Forks
1
PR merge metrics
No merged PRs in 30d

Description

**Bug report**

Back-end filtering works for numeric quantities but not for string/enum quantities. Both of these attempts fail:

```python
gaia3 = lsdb.read_hats(
"http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
filters=[
["phot_variable_flag", "=", "CONSTANT"], # bare string
],
)
gaia3 = lsdb.read_hats(
"http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
filters=[
["phot_variable_flag", "=", '"CONSTANT"'], # forced string literal
],
)
```

Both of these cause this back-end error from lsdb-server:
```
called `Result::unwrap()` on an `Err` value: NotYetImplemented("Data type Utf8 not yet implemented")
```

The first approach works with `https://data.lsdb.io/hats/gaia_dr3/gaia`, and the second does not (and perhaps should not).

*NOTE*: there are no columns in Gaia DR3 where `phot_variable_flag` has the value `CONSTANT`. There is only `NOT_AVAILABLE` and `VARIABLE`; be aware of that when testing.

**Before submitting**
Please check the following:

- [X] I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
- [X] I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
- [ ] If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the issue using the Python lsdb.read_hats examples and the Gaia DR3 values NOT_AVAILABLE or VARIABLE, then trace the lsdb-server filtering path that handles string columns. Done means bare string filters work without the "Data type Utf8 not yet implemented" error while numeric filtering continues to work.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.