apache / apache/hertzbeat

[Discuss] <Large Language Model (LLM) systems Metrics>

Open
#2,273 0 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Java
Stars
7.4k
Forks
1.3k
Avg merge
1d 20h
Merged PRs (30d)
32

Description

### Discuss

monitoring llm metrics and more.

https://medium.com/data-science-at-microsoft/evaluating-llm-systems-metrics-challenges-and-best-practices-664ac25be7e5

image

Contributor guide

Open the contributing guide

Research direction

Start by reading the linked article and the repository's existing monitoring and metrics architecture; the issue names no files, tests, or entry points. Before implementation, define which LLM metrics are needed, where they should be collected, and what validation will demonstrate that the monitoring is complete.

Written by the indexing model from the issue text.

Assessment

Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.