astronomy-commons / astronomy-commons/lsdb-server
lsdb-server cannot filter based on string columns
- Dominant language
- Rust
- Stars
- 2
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
**Bug report**
Back-end filtering works for numeric quantities but not for string/enum quantities. Both of these attempts fail:
```python
gaia3 = lsdb.read_hats(
"http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
filters=[
["phot_variable_flag", "=", "CONSTANT"], # bare string
],
)
gaia3 = lsdb.read_hats(
"http://epyc.astro.washington.edu:43210/hats/gaia_dr3/gaia/",
filters=[
["phot_variable_flag", "=", '"CONSTANT"'], # forced string literal
],
)
```
Both of these cause this back-end error from lsdb-server:
```
called `Result::unwrap()` on an `Err` value: NotYetImplemented("Data type Utf8 not yet implemented")
```
The first approach works with `https://data.lsdb.io/hats/gaia_dr3/gaia`, and the second does not (and perhaps should not).
*NOTE*: there are no columns in Gaia DR3 where `phot_variable_flag` has the value `CONSTANT`. There is only `NOT_AVAILABLE` and `VARIABLE`; be aware of that when testing.
**Before submitting**
Please check the following:
- [X] I have described the situation in which the bug arose, including what code was executed, information about my environment, and any applicable data others will need to reproduce the problem.
- [X] I have included available evidence of the unexpected behavior (including error messages, screenshots, and/or plots) as well as a description of what I expected instead.
- [ ] If I have a solution in mind, I have provided an explanation and/or pseudocode and/or task list.
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the issue using the Python lsdb.read_hats examples and the Gaia DR3 values NOT_AVAILABLE or VARIABLE, then trace the lsdb-server filtering path that handles string columns. Done means bare string filters work without the "Data type Utf8 not yet implemented" error while numeric filtering continues to work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100