google/metrax

View on GitHub

A JAX-native High Performance Eval Metrics Library

Stars: 64
Forks: 18
Open beginner issues: 0
Indexed issues: 19
Avg merge: 4h 46m
Merged PRs (30d): 1

Dominant language: Python
License: Apache-2.0
Last GitHub push: Jul 22, 2026
Latest indexed: Sep 16, 2026
Contributing guide: Contributing guide
Code of conduct: Code of conduct
Beginner labels: good first issue help wanted

Open All

19 open issues indexed

Search for custom subclassed CLU metrics in popular repos and send them PRs to upgrade Open

good first issue

google/metrax#3 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Add recommendation for how to run distributed eval Open

google/metrax#18 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Establish parity with TFRS Open

google/metrax#19 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Add FactorizedTopK metric Open

google/metrax#20 · 0 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Add Mean Intersection over Union (mIoU) metric Open

good first issue help wanted

google/metrax#61 · 3 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Add Total Variation (TV) Open

good first issue help wanted

google/metrax#63 · 3 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Add Kernel Inception Distance (KID) metric Open

good first issue help wanted

google/metrax#65 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Add advanced tokenizer for LLM metrics Open

google/metrax#68 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Investigate if RecML can use Metrax metrics Open

google/metrax#69 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Merge global test variables into a helper file Open

google/metrax#89 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Makefile for local development Open

google/metrax#96 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Failing Unit Tests Open

google/metrax#101 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Add from_logits flag to every Metrax metric Open

good first issue help wanted

google/metrax#105 · 4 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Unpin JAX version after next jax_tpu_embedding release Open

google/metrax#107 · 2 comments · 0 reactions · 1 assignee · Indexed Sep 16, 2026
Update dev dependencies to Python 3.13 Open

google/metrax#115 · 4 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
`FBetaScore` missing `merge` method causes NotImplementedError with `metrax.nnx` wrapper Open

google/metrax#131 · 2 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026
`metrax.nnx` metrics incompatible with `nnx.MultiMetric` due to strict `kwargs` in `from_model_output` Open

google/metrax#132 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Make codebase compatible with `ruff format` Open

google/metrax#140 · 1 comment · 0 reactions · 0 assignees · Indexed Sep 16, 2026
Mention how to use NNX metrics in the docs Open

google/metrax#146 · 0 comments · 0 reactions · 0 assignees · Indexed Sep 16, 2026

google/metrax

Get new issues in your inbox