JetBrains-Research / JetBrains-Research/opaque

Sweep D7 — alignment and auditing residue

Open
#1,069 0 comments 0 reactions 0 assignees View on GitHub
impact: api impact: numerical pkg: alignment pkg: auditing severity: medium source: audit
Dominant language
Python
Stars
3
Forks
0
Avg merge
15h 4m
Merged PRs (30d)
187

Description

## Problem

Eight defects across `opaque-alignment` and `opaque-auditing`, batched for one owner because several are residue of the same two earlier fixes.

### The #895 dataset guard — two halves still open, do them together

PR #895 added a dataset guard to the auditing scoring path (the fix closed as #830). It is half a guard.

| Where | What is wrong | What to do |
| --- | --- | --- |
| `auditing/_coin_flip.py:189, 209` | The guard covers the **scoring** half only: `train_subset()` / `train_indices()` still accept a dataset the `CoinFlip` was not built from | In `train_subset`, either drop the explicit argument (call `self.train_indices()`) or compare `len(dataset)` against `self.dataset_size` and raise the same `ConfigurationError` |
| `auditing/attacks/_helpers.py:152-178` | The scoring guard compares dataset **length only**, so an equal-length reordering is still scored silently — and the original defect was precisely indices resolving against the wrong dataset while looking plausible | Record a cheap fingerprint at `coin_flip()` time — blake2b over the bytes/repr of a handful of canary rows from the dataset it was constructed on — and compare that, not just the length |

A test should score against a **same-length but different** dataset and assert it raises. Length equality is a heuristic; identity is a guard.

### Chat-template handling

| Where | What is wrong | What to do |
| --- | --- | --- |
| `alignment/data/_chat_template.py:326-332` | `get_training_chat_template` **silently changes the rendered conversation text**, because Jinja's `trim_blocks` eats the newline the inserted block introduces | Make `_generation_block_marks_only_assistant` reject any candidate whose full rendered string differs from the untransformed template's rendering for the same probe conversation |
| `alignment/data/_completion_mask.py:114-124` | `apply_chat_template_with_mask` reports a **per-row truncation** as a chat-template misconfiguration and aborts the whole dataset | Split the two conditions: keep the template check as `ConfigurationError`; for an empty mask on a rendered row raise a distinct error naming truncation and the row |
| `alignment/sft/collator/_language_modeling.py:147-161` | `language_modeling_collator(completion_only_loss=True)` **supervises the full sequence** for maskless rows — the opposite of what the flag requests — and fabricates a mask | Keep the TRL-matching default but make it explicit rather than silent: when the flag is set and a row supplies no `completion_mask`, raise, or warn once naming the row count |
| `alignment/tests/data/test_chat_template.py:379-383` | A parametrized test labels its cases `gemma-3-1b-it` and `shared-chatml` while exercising **neither** | Either rename the ids to the shapes actually exercised (`shared-set-content`, `inline-role-mapping`, `explicit-assistant-branch`), or make the named templates real. A test whose id names a template it does not run is worse than no test — it reads as coverage |

### Auditing documentation and comments

| Where | What is wrong | What to do |
| --- | --- | --- |
| `auditing/one_run/_estimate.py:34-42` | The grid-floor justification comment records a measurement that **does not reproduce**, and cites a test validating something else | Restate with the configuration the regression test actually uses: 500/500 with 250 errors at the best threshold gives ε 0.0 at grid 16, 2.70 at 64, 5.08 at 1000 |
| `docs/reference/auditing.md:92-94, 148-150, 238-249` | #895's new public field and new failure mode never reached the reference | Add `dataset_size` to the `CoinFlip` attribute table, show `train_indices(dataset_size=None)` with the recorded-size default, extend both scorers' Raises blocks. (Also listed in #1064 — do it once, in whichever lands first) |

## Acceptance criteria

- The two #895 guard halves land as **one** change, and the scoring guard compares identity rather than length.
- A test scores against a same-length different dataset and asserts it raises.
- No chat-template transform alters the rendered conversation text; a test compares full renderings before and after.
- `completion_only_loss=True` never silently supervises a full sequence.
- Every test id names the shape it actually exercises.
- Each row is fixed or declined with a reason recorded here.

Found in the September 2026 library audit.

Contributor guide

Open the contributing guide

Research direction

Start with the named auditing files and alignment modules, then run the referenced chat-template tests and inspect the documentation sections. Trace the existing dataset guards, rendering checks, masks, collator behavior, test parametrization, and estimate comment before changing them. Done means each listed defect is fixed or declined with a reason, with regression tests covering the specified cases and documentation updated.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
documentation, machine-learning, testing-qa
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.