explosion / explosion/spacy-llm

Potential REL sharding issue

Open
#450 3 comments 1 reaction 0 assignees View on GitHub
bug feat/sharding feat/task
Dominant language
Python
Stars
1.4k
Forks
110
PR merge metrics
No merged PRs in 30d

Description

I use spacy-llm for RAG, so I have integrated pretty deeply into it and 0.7.0 broke my method of prompt generation. In the process of fixing it, I've noticed something that may be a more general bug.

The REL shard reducer [unpacks RelationItem objects from each shard into a single vector](https://github.com/explosion/spacy-llm/blob/c87d5a6373485c7510d92b5fed08770f82c96c1a/spacy_llm/tasks/rel/util.py#L95-L100).

The comment in the code says this is fine, but RelationItem objects include the entity index of the dep/dest of the relationship. I think that's going to be 0-based for each shard, so the unpacked list will not have the right relationships for all shards after the first.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.