ai-cfia / ai-cfia/ailab-db

Search functions return wrong content in `ailab-db`

オープン
#91 コメント 0 件 リアクション 1 件 担当者 1 名 @k-allagbe が担当を希望しています GitHub で見る
主要言語
Jupyter Notebook
スター
1
フォーク
0
PR マージ指標
30日以内にマージされた PR はありません

説明

## Description

Search functions now return funny values for the `content` field.

```python
def search(cursor, query_embedding):
"""Search matching documents with a given query and return a dict."""
data = {
'text': ' '.join(sys.argv[1:]),
'query_embedding': query_embedding,
'match_threshold': 0.5,
'match_count': 10,
'weights': json.dumps(FINESSE_JSON_PARSED_WEIGHTS)
}

cursor.execute("""
SELECT *
FROM search(%(text)s, %(query_embedding)s::vector, %(match_threshold)s,
%(match_count)s, %(weights)s::JSONB)
""", data)
# turn into list of dict now to preserve dictionaries
results = cursor.fetchall()
from pprint import pprint
pprint(results)
return [dict(r) for r in results[0]["search"]]
```
Printed value for "who's the president?"
```json
[
{
"search": [
{
"content": "2004",
"id": "1bc6def7-339e-49cf-996c-8559c8f074b0",
"last_updated": "2022-08-02",
"query_id": "b685ff7a-7bc3-40c7-88de-8f6183a13f2a",
"score": 0.43899683490982105,
"scores": {
"current": 1,
"recency": 0.8294036718602102,
"similarity": 0.778211130010486,
"traffic": 0.08711416212970638,
"typicality": 0.0012760527435133986,
},
"subtitle": "2004",
"title": "L'ACIA : beaucoup de chemin parcouru depuis 1997 - "
"Agence canadienne d'inspection des aliments",
"url": "https://inspection.canada.ca/inspecter-et-proteger/salubrite-des-aliments/chemin-parcouru/fra/1655235733366/1655496070310",
},
{
"content": "2004",
"id": "9a8ba37c-8b15-4b3f-9f5d-49ca50f92d35",
"last_updated": "2022-08-02",
"query_id": "b685ff7a-7bc3-40c7-88de-8f6183a13f2a",
"score": 0.43874722259501314,
"scores": {
"current": 1,
"recency": 0.8294036718602102,
"similarity": 0.778211130010486,
"traffic": 0.08586610055566671,
"typicality": 0.0012760527435133986,
},
"subtitle": "2004",
"title": "L'ACIA : beaucoup de chemin parcouru depuis 1997 - "
"Agence canadienne d'inspection des aliments",
"url": "https://inspection.canada.ca/inspecter-et-proteger/sante-des-vegetaux/chemin-parcouru/fra/1655235733366/1655496024138",
}
]
}
]

```

1. The chunk's full text or snippet is expected to be received in the `content` field.
2. The subtitle values also look funny, although I'm not sure what is supposed to be received there.
3. I suggest we receive the value (list of results) of the `search` field instead of `[ { "search": [...] } ]`

For comparison we receive this from azure-db:
```json
[
{
"content": "Kochhar\n \n \n CFIA President\n Dr.",
"id": "ZGI2M2VmNjktNTVmMC00ODQ2LThlZWItZDljYTYwZDMwNTI10",
"last_updated": "2023-04-18T00:00:00Z",
"score": 8.465198,
"title": "Dr. Harpreet S. Kochhar - Canadian Food Inspection Agency",
"url": "https://inspection.canada.ca/about-cfia/organizational-structure/cfia-president/eng/1681496883837/1681496884212",
},
{
"content": "The CFIA is headed by a "
"President, who has the rank "
"and all the powers of a Deputy Head of a "
"Department.",
"id": "ZDg1YzNlMjEtMzhjMS00NTQ2LWFhYTEtM2ZjOWUyOTZmZWFm0",
"last_updated": "2017-08-28T00:00:00Z",
"score": 8.322858,
"title": "Canadian Food Inspection Agency (CFIA) - Quarterly Financial "
"Report (QFR) for the Quarter ended June 30, 2017 - Canadian Food "
"Inspection Agency",
"url": "https://inspection.canada.ca/about-cfia/transparency/corporate-management-reporting/reports-to-parliament/financial-reporting/quarter-ended-june-30-2017/eng/1502989987656/1502989988316",
},
]
```

## Acceptance criteria
- Full text or relevant snippet is received in the `content` field

コントリビューションガイド

このリポジトリのコントリビューションガイドは索引されていません

評価

この issue はまだ評価されていません。

新しい issue をメールで受け取る

初心者向けの GitHub issue を短くまとめたダイジェスト。