AOSSIE-Org / AOSSIE-Org/EduAid

Update deprecated pad_to_max_length argument in tokenization pipeline

Ouverte
#511 2 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
JavaScript
Étoiles
171
Forks
423
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

## Description

While running EduAid locally, warnings appear from the HuggingFace Transformers library related to deprecated parameters used in the tokenization pipeline.

The following warning is shown during execution:

FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in future versions.

The current implementation uses:

pad_to_max_length=True

This parameter has been deprecated and should be replaced with the newer argument format recommended by the Transformers library.

## Expected Behavior

Tokenization should use the updated padding parameter recommended by the Transformers library.

## Current Behavior

The code currently uses deprecated arguments which trigger warnings during execution.

## Proposed Improvement

Replace the deprecated parameter:

pad_to_max_length=True

with the updated format:

padding='max_length'

This will align the codebase with the latest Transformers API.

## Benefits

• removes deprecation warnings
• ensures compatibility with future Transformers versions
• improves long-term maintainability of the codebase

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.