AOSSIE-Org / AOSSIE-Org/EduAid

Update deprecated pad_to_max_length argument in tokenization pipeline

Abierto
#511 2 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
JavaScript
Estrellas
171
Forks
423
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Description

While running EduAid locally, warnings appear from the HuggingFace Transformers library related to deprecated parameters used in the tokenization pipeline.

The following warning is shown during execution:

FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in future versions.

The current implementation uses:

pad_to_max_length=True

This parameter has been deprecated and should be replaced with the newer argument format recommended by the Transformers library.

## Expected Behavior

Tokenization should use the updated padding parameter recommended by the Transformers library.

## Current Behavior

The code currently uses deprecated arguments which trigger warnings during execution.

## Proposed Improvement

Replace the deprecated parameter:

pad_to_max_length=True

with the updated format:

padding='max_length'

This will align the codebase with the latest Transformers API.

## Benefits

• removes deprecation warnings
• ensures compatibility with future Transformers versions
• improves long-term maintainability of the codebase

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.