NVIDIA-NeMo / NVIDIA-NeMo/Anonymizer

feat(rewrite): revisit HIGH sensitivity classification and any_high_leaked trigger for needs_human_review

Open
#107 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement high priority
Dominant language
Python
Stars
122
Forks
17
Avg merge
4d 2h
Merged PRs (30d)
11

Description

Background

needs_human_review is triggered by two conditions: leakage_mass > flag_leakage_mass_above (default 2.0) or any_high_leaked=True. Investigation of TAB runs showed all flagged rows were triggered by any_high_leaked, not leakage mass.

The entities flagged as HIGH sensitivity were:

  • court_name (e.g. 'District Court', 'Budapest District Court') — fixed by PR #94 domain supplements
  • legal_role (e.g. 'defendant_in_recovery_action')
  • former_senator (occupation type)

These are quasi-identifiers at most. Flagging them as HIGH causes needs_human_review=True even when leakage mass is low and utility is good (e.g. utility=0.97, leakage=0.96).

Questions to resolve

  • What is the right criteria for HIGH vs. MEDIUM sensitivity? Should quasi-identifrs ever be HIGH?
  • Should any_high_leaked alone be sufficient to trigger needs_human_review, or should it require a minimum leakage mass too?
  • Should sensitivity level be configurable per domain (e.g. LEGAL may have different thresholds than MEDICAL)?

Related

  • PR #94 partially addressed this for court_name via LEGAL domain supplements
  • Ties into the domain-specific risk tolerance work removed before release (see domain supplement follow-up issue)

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Review the implementation of sensitivity classification and the needs_human_review trigger, then compare it with PR #94 and the domain supplement follow-up issue. Resolve the HIGH versus MEDIUM criteria, the role of any_high_leaked, and whether domain-specific thresholds are needed. Done means the policy is agreed and reflected in the relevant behavior and validation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.