SearchIndex.from_existing silently drops vector fields on a RESP2 connection with legacy_responses=False
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 427
- Forks
- 101
- Avg merge
- 6d 3h
- Merged PRs (30d)
- 20
Description
Round-tripping an index through SearchIndex.from_existing and then create() rebuilds it without its vector field, silently, when the supplied client is a RESP2 connection created with redis-py 8's legacy_responses=False. Nothing is logged and nothing is raised. For a vector search library this is the worst available failure mode: the rebuilt index reports itself as healthy, FT.INFO lists it as built, and every vector query against it is searching for a field that no longer exists.
Reproduction
Measured on Redis 8.4.6 with redis-py 8.1.0, RedisVL at edcc78e. The source index has three fields, one of them a FLAT vector field named emb.
from redis import Redis
from redisvl.index import SearchIndex
client = Redis.from_url("redis://localhost:6379", protocol=2, legacy_responses=False)
recovered = SearchIndex.from_existing("my_index", redis_client=client)
print(sorted(recovered.schema.field_names)) # ['cat', 'text'], and 'emb' is gone
recovered.schema.index.name = "my_index_rebuilt"
recovered.create(overwrite=True)
FT.INFO my_index_rebuilt then reports attributes [b'text', b'cat']. The rebuilt index really does lack the vector field; that is a measured result rather than an inference from the schema.
Cause
redis-py's two unified FT.INFO parsers disagree about where vector parameters live. On a RESP3 connection they are promoted to top-level keys, measured on the same index as:
{b'identifier': b'emb', b'attribute': b'emb', b'type': b'VECTOR',
b'algorithm': b'FLAT', b'data_type': b'FLOAT32', b'dim': 4,
b'distance_metric': b'COSINE', b'flags': []}
On a RESP2 connection with legacy_responses=False, _parse_info_unified packs the same parameters into the flags list instead:
{'identifier': 'emb', 'attribute': 'emb', 'type': 'VECTOR',
'flags': ['algorithm', 'FLAT', 'data_type', 'FLOAT32', 'dim', 4,
'distance_metric', 'COSINE']}
parse_vector_attrs in redisvl/redis/connection.py takes its dict branch here and excludes flags from the keys it copies (line 321), so vector_attrs comes out empty. dims is therefore absent, and because dims is required the function returns None. The caller at line 548 reads that None as "vector attributes cannot be parsed on this Redis version" and does a bare continue at line 551, dropping the field from the schema without a word.
The continue is defensible on its own terms, since it was written for Redis 6.2.6-v9 where FT.INFO genuinely returns no vector parameters. What is not defensible is that it now also absorbs a parse failure on a modern server where the parameters were present in the reply all along.
Scope
Measured across three client configurations against the same index:
| Client | Fields recovered by from_existing |
|---|---|
protocol=2 (RedisVL's own default) |
['emb', 'text'] |
protocol=2, legacy_responses=False |
['text'] |
protocol=3 |
['emb', 'text'] |
Only the middle row is affected, so a fix should target the RESP2 unified shape rather than RESP3 handling, which already works. Note that legacy_responses is a redis-py 8 only keyword; on 6.3.0 and 7.4.0 it raises TypeError from AbstractConnection.__init__ at connect time, so this configuration is reachable only on redis-py 8.
Suggested fix
Two changes, independently useful. First, teach the dict branch of parse_vector_attrs to unpack a non-empty flags list as alternating key-value pairs before excluding it, which restores dims and the rest of the parameters. Second, replace the bare continue with a warning naming the field and the index, so that a future parse gap degrades loudly instead of silently. The second is worth doing regardless of the first: any path that yields a schema missing a field the server reported should say so.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in redisvl/redis/connection.py at parse_vector_attrs and the caller around lines 548-551; inspect the RESP2 unified FT.INFO shape and the existing vector parsing branches. Reproduce with Redis 8.4.6, redis-py 8.1.0, protocol=2, and legacy_responses=False. Done means from_existing preserves the emb vector field, create() rebuilds it, and unparseable fields produce a warning naming the field and index.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, redis
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100