AlibabaResearch / AlibabaResearch/DAMO-ConvAI

Use of Set Conversion for SQL Query Results Comparison in execute_sql Function

Offen
#190 0 Kommentare 1 Reaktion 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Python
Sterne
1.6k
Forks
250
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Issue Description:

There is an incorrect use of set conversion for SQL query results comparison in the execute_sql function within the evaluation.py file. The current implementation compares the predicted SQL results with the ground truth SQL results using sets, which removes duplicates and disregards the order of the results. This can lead to incorrect evaluation results when duplicates are important or when the order of the results matters.

Expected Behavior:

The function should compare the results of the SQL queries directly without converting them to sets, to preserve duplicates and the order of the results.

Suggested Fix:

Replace the line of code that converts the results to sets with a direct comparison of the lists:

```
if predicted_res == ground_truth_res:
res = 1
```

This ensures that the evaluation considers both duplicates and the order of the results.

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.