ai-cfia / ai-cfia/api-test

Enhance Finesse Benchmark Tool with Cost Estimation and Data Visualization Features

Offen
#1 1 Kommentar 0 Reaktionen 1 zugewiesene Person Beansprucht von @elwoujdi Auf GitHub ansehen
epic
Vorherrschende Sprache
Python
Sterne
1
Forks
0
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

## Summary
The finesse benchmark tool currently provides an accuracy score and request time for each JSON file. However, it lacks the capability to generate cost estimates. This tool is responsible for comparing different search engines to discern the best from the worst, and one decisive factor is inevitably the cost incurred by using a tool. Additionally, the tool saves tests locally, and not all the test data is exported to CSV and MD files, such as the entirety of search engine responses. This can result in loss of results and traceability. Finally, it would be beneficial to add a data visualizer such as a graph. Therefore, we are enhancing this tool with the locust-dashboard, which enables us to implement all the aforementioned features.

## Tasks:
- [ ] #4
- [ ] Integrate cost estimation feature into finesse benchmark tool.
- [ ] Create Jupyter Notebooks to save data on md and csv file
- [ ] Implement a data visualizer on the locust-dashboard, such as a graph, to enhance data analysis capabilities.

## Acceptance Criteria
- The finesse benchmark tool successfully generates cost estimates for each search engine.
- All test data, including search engine responses, are saved on datahub
- The implemented data visualizer effectively presents the benchmark results, enhancing data analysis capabilities.

## Diagram

```mermaid
sequenceDiagram
User ->> Finesse_Tool: Start test
Finesse_Tool ->> Finesse_Tool: Cost estimate
Finesse_Tool ->> Datahub: Save test data
Datahub ->> Locust_Dashboard: Retrieve data
Locust_Dashboard -->> User: Visualization displayed
Datahub ->> Jupyter_Notebook: Retrieve data
Jupyter_Notebook -->> Finesse_Tool: Generate md or csv file
```

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.