huggingface / huggingface/setfit

Dealing with large documents

Open
#275 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
2.8k
Forks
267
Avg merge
36m
Merged PRs (30d)
5

Description

Hi, thanks for making the library so usable! Asking for suggestions.

**Context:**

- Running SetFit on a small set of documents (financial; ~40 docs for 4 classes; ~10 docs per class for testing)
- The documents in question are quite big (~10k+ characters), so only the beginning of the doc is used
- Regardless, the performance is acceptable

**Problem:**

Analyzing the results, for e.g. with SHAP, its clear that we can squeeze better performance using the entire document.

**Potential Approach:**

Simplest idea is to break the document into chunks and train with those chunks. Run inference for all chunks and take a vote to decide the final class. An improvement would be to train a different "head" - that can take multiple chunks together (fixed number, as want to avoid LSTM).

**Any experience / suggestions on this will greatly help :)**

Contributor guide

No contributing guide indexed for this repository

Research direction

The issue discusses handling 10k+ character documents in SetFit, including chunking, voting across chunks, and a multi-chunk classification head. No source files, tests, or entry points are identified; first clarify the desired approach and requirements before locating the relevant training and inference code. Done would require an agreed design and implementation with tests for large-document classification.

Written by the indexing model from the issue text.

Assessment

Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.