Azure/azure-openai-benchmark

Azure OpenAI benchmarking tool

Stars: 154
Forks: 76
Open beginner issues: 0
Indexed issues: 23

Dominant language: Python
License: MIT
Last GitHub push: Mar 14, 2024
Latest indexed: Sep 15, 2026
Contributing guide: No contributing guide
Code of conduct: Code of conduct
Beginner labels: No beginner labels indexed
PR merge metrics: No merged PRs in 30d

Open All

23 open issues indexed

Question about using wonderwords to generate random prompt info Open

enhancement

Azure/azure-openai-benchmark#9 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Is it possible to do benchmark testing on APIM which has couple of Azure Open AI resources in backend? Open

Azure/azure-openai-benchmark#22 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Any plan to adding visualization result in this package? Open

Azure/azure-openai-benchmark#23 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Benchmark tool Output to CSV Open

Azure/azure-openai-benchmark#24 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Utilization Statistics - avg, P95 Open

Azure/azure-openai-benchmark#25 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Typo when setting "presence penalty" parameter Open

Azure/azure-openai-benchmark#28 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
RequestBuilder is prepared with hardcoded value of "gpt-4-0613" Open

Azure/azure-openai-benchmark#29 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Renaming "frequence_penalty" function argument to "frequency_penalty" Open

Azure/azure-openai-benchmark#30 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Considering "retry-after" header as well when retrying throttled requests. Open

Azure/azure-openai-benchmark#31 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Documenting reference for header values used. Open

Azure/azure-openai-benchmark#32 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
RETRY_AFTER_MS_HEADER not getting properly "honored" Open

Azure/azure-openai-benchmark#33 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Decoupling Throttling retries logic & Throttling backoff logic. Open

Azure/azure-openai-benchmark#34 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Value for MAX_RETRY_SECONDS should not be hardcoded Open

Azure/azure-openai-benchmark#36 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Documenting about MAX_RETRY_SECONDS Open

Azure/azure-openai-benchmark#37 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Documenting about the two types of retry logics for 429 errors. Open

Azure/azure-openai-benchmark#38 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Count of Completed Requests & Total Requests is same Open

Azure/azure-openai-benchmark#40 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Properly reflecting the count of "requests" Open

Azure/azure-openai-benchmark#41 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Purpose of stats call condition in while loop? Open

Azure/azure-openai-benchmark#42 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Difference in the e2e_avg latency observed in the benchmarking tool and Azure portal Open

Azure/azure-openai-benchmark#53 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Issue with azure benchmarking tool on passing input tokens greater than 30k Open

Azure/azure-openai-benchmark#54 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
Issue while running benchmarking tool in setting the rpm Open

Azure/azure-openai-benchmark#55 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026
[FEAT] Embeddings Benchmark Open

Azure/azure-openai-benchmark#56 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 15, 2026
The thread cannot be completely stopped, and the test time is up, but it is still running Open

Azure/azure-openai-benchmark#57 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 15, 2026

A short digest of beginner-friendly GitHub issues.