AOSSIE-Org / AOSSIE-Org/EduAid

Update deprecated pad_to_max_length argument in tokenization pipeline

Open
#511 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
171
Forks
423
PR merge metrics
No merged PRs in 30d

Description

## Description

While running EduAid locally, warnings appear from the HuggingFace Transformers library related to deprecated parameters used in the tokenization pipeline.

The following warning is shown during execution:

FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in future versions.

The current implementation uses:

pad_to_max_length=True

This parameter has been deprecated and should be replaced with the newer argument format recommended by the Transformers library.

## Expected Behavior

Tokenization should use the updated padding parameter recommended by the Transformers library.

## Current Behavior

The code currently uses deprecated arguments which trigger warnings during execution.

## Proposed Improvement

Replace the deprecated parameter:

pad_to_max_length=True

with the updated format:

padding='max_length'

This will align the codebase with the latest Transformers API.

## Benefits

• removes deprecation warnings
• ensures compatibility with future Transformers versions
• improves long-term maintainability of the codebase

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.