NeuroTechX / NeuroTechX/moabb

Allow a metadata-aware splitter as cv_class (used directly, not wrapped as the inner groups-CV)

Open
#1,104 1 comment 1 reaction 2 assignees View on GitHub

@bruAristimunha is already working on this.

Since Jun 27, 2026.

Dominant language
Python
Stars
1.1k
Forks
264
Avg merge
1d 13m
Merged PRs (30d)
23

Description

Summary

CrossSubjectEvaluation (and the CrossSubjectSplitter/CrossSessionSplitter wrappers)
always treat cv_class as the inner CV: they instantiate it and call
split(X=index, y, [groups=subject]), forwarding only the subject column. There is no
public way to plug in a metadata-aware top-level splitter — one that needs split(y, metadata)
and other columns (e.g. session) to build the fold.

Why this matters

We want a single-target cross-subject fold that restricts the test set to one session while
excluding the target's other sessions from training (test = target@session, train = other
subjects). That decision needs the session column, which the inner-CV path never passes.

A splitter that already follows the moabb convention (subclasses BaseCrossValidator, declares
metadata_columns = ("subject", "session"), implements split(self, y, metadata)) cannot be
passed via cv_class:

import pandas as pd
from moabb.evaluations.splitters import CrossSubjectSplitter
# MySplitter: BaseCrossValidator with metadata_columns=("subject","session") and split(self, y, metadata)
md = pd.DataFrame({"subject": [1, 1, 2, 2], "session": ["0", "1", "0", "1"]})
list(CrossSubjectSplitter(cv_class=MySplitter, target=2, test_session="1").split(y=None, metadata=md))
# TypeError: MySplitter.split() got an unexpected keyword argument 'X'
Current workaround

Subclass the evaluation and override _create_splitter to use cv_class directly:

class TargetSubjectEvaluation(CrossSubjectEvaluation):
    def _create_splitter(self):
        return self.cv_class(**self.cv_kwargs) if self.cv_class else super()._create_splitter()
Proposed

The evaluation already inspects cv_class.metadata_columns in __init__ (to extend
additional_columns). Extend that: when cv_class declares metadata_columns, use it directly
as the top-level splitter — call split(y, metadata) — instead of wrapping it as the inner
groups-CV. Backward compatible (group CVs without metadata_columns keep wrapping), and it lets
metadata-driven folds be configured through the public cv_class / cv_kwargs API with no subclass.

Happy to open a PR if this direction sounds good.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.