microsoft / microsoft/PyRIT

Labelled responses from remote datasets can't reach `HumanLabeledDataset`

Open
#2,475 11 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
4.5k
Forks
893
Avg merge
3d 50m
Merged PRs (30d)
165

Description

There's no path from the remote dataset loaders to the labelled-data side of scoring.

**What's there now.** `HumanLabeledDataset` is only constructible via `from_csv()`, so scorer evaluation data has to be authored by hand. That's reflected in `datasets/scorer_evals/` — `refusal.csv` is ~105 rows and several of the objective sets are single digits.

**What's being dropped.** `datasets/seed_datasets/remote/` has 61 loaders, and several of the underlying HuggingFace datasets ship human labels alongside model responses. Those are discarded at load time because `SeedDataset` models prompts only. `beaver_tails_dataset.py` states it directly:

> This loader extracts only the prompts (not the responses) and filters to unsafe entries by default.

`aegis_ai_content_safety_dataset.py` is the same shape. `wildguardmix_dataset.py` at least retains its classifier labels in `metadata`, but still produces a `SeedDataset`.

So labelled `(response, score)` pairs are already flowing through the loaders and being dropped, while the one class that needs them can only be fed by hand.

**Proposal.** A second output path off the existing remote loaders producing `HumanLabeledDataset` — reusing the fetch logic and the `HARM_CATEGORY_ALIAS_OVERRIDES` mapping that's already there, but retaining `assistant_response` and the human label.

Suggest starting narrow: Aegis 2.0 only (CC-BY-4.0, so no licence friction for a repo shipping under MIT) and one harm category that maps cleanly, with tests and a generated `scorer_evals` CSV so it's usable on merge.

Happy to build it if the shape is right — is `HumanLabeledDataset` the correct target here, or is there a reason the loaders are prompts-only that I'm missing?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with datasets/seed_datasets/remote/aegis_ai_content_safety_dataset.py and the HumanLabeledDataset path, then compare the existing HARM_CATEGORY_ALIAS_OVERRIDES mapping and loader behavior with datasets/scorer_evals/refusal.csv. Add focused tests for the narrow Aegis 2.0 case and generate a usable scorer_evals CSV retaining assistant_response and the selected human label.

Written by the indexing model from the issue text.

Assessment

Tech stack
huggingface, python
Domain
data, machine-learning, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.