Failed to query string column in noDictionaryColumns
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 195
Description
A simple select * query leads to query failure with following. This happens only for a few rows and only in offline segments. The column is single value string with raw forward encoding
```
...
...
"noDictionaryColumns": [
"compressed_data_string"
]
},
"fieldConfigList": [
{
"name": "compressed_data_string",
"encodingType": "RAW",
"indexTypes": [],
"properties": {
"deriveNumDocsPerChunkForRawIndex": "true",
"rawIndexWriterVersion": "3"
}
}
]
```
Error trace:
```
Caught exception while processing query
java.io.IOException: FAILED_TO_UNCOMPRESS(5)
at org.xerial.snappy.SnappyNative.throw_error(SnappyNative.java:112)
at org.xerial.snappy.SnappyNative.rawUncompress(SnappyNative.java)
at org.xerial.snappy.Snappy.uncompress(Snappy.java:551)
at o.a.p.s.l.i.c.SnappyDecompressor.decompress(SnappyDecompressor.java:40)
at o.a.p.s.l.s.i.r.f.BaseChunkForwardIndexReader.decompressChunk(BaseChunkForwardIndexReader.java:140)
... 25 common frames omitted
Wrapped by: java.lang.RuntimeException: java.io.IOException: FAILED_TO_UNCOMPRESS(5)
at o.a.p.s.l.s.i.r.f.BaseChunkForwardIndexReader.decompressChunk(BaseChunkForwardIndexReader.java:143)
at o.a.p.s.l.s.i.r.f.BaseChunkForwardIndexReader.getChunkBuffer(BaseChunkForwardIndexReader.java:121)
at o.a.p.s.l.s.i.r.f.VarByteChunkSVForwardIndexReader.getStringCompressed(VarByteChunkSVForwardIndexReader.java:79)
at o.a.p.s.l.s.i.r.f.VarByteChunkSVForwardIndexReader.getString(VarByteChunkSVForwardIndexReader.java:68)
at o.a.p.s.l.s.i.r.f.VarByteCh...
```
Contributor guide
Research direction
Start with BaseChunkForwardIndexReader.decompressChunk and getChunkBuffer, then inspect VarByteChunkSVForwardIndexReader.getStringCompressed and the SnappyDecompressor path shown in the trace. Reproduce a simple select * query against an offline segment containing the RAW single-value string column and determine why only some rows fail; done means the query no longer raises FAILED_TO_UNCOMPRESS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100