AOSSIE-Org / AOSSIE-Org/EduAid

Update deprecated pad_to_max_length argument in tokenization pipeline

未关闭
#511 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
主要语言
JavaScript
星标
171
派生
423
PR 合并指标
30 天内没有已合并 PR

描述

## Description

While running EduAid locally, warnings appear from the HuggingFace Transformers library related to deprecated parameters used in the tokenization pipeline.

The following warning is shown during execution:

FutureWarning: The `pad_to_max_length` argument is deprecated and will be removed in future versions.

The current implementation uses:

pad_to_max_length=True

This parameter has been deprecated and should be replaced with the newer argument format recommended by the Transformers library.

## Expected Behavior

Tokenization should use the updated padding parameter recommended by the Transformers library.

## Current Behavior

The code currently uses deprecated arguments which trigger warnings during execution.

## Proposed Improvement

Replace the deprecated parameter:

pad_to_max_length=True

with the updated format:

padding='max_length'

This will align the codebase with the latest Transformers API.

## Benefits

• removes deprecation warnings
• ensures compatibility with future Transformers versions
• improves long-term maintainability of the codebase

贡献指南

这个仓库没有索引到贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。