asyml / asyml/forte

Allow users assign specific index for PAD or UNK in Vocabulary

Open
#414 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
253
Forks
59
PR merge metrics
No merged PRs in 30d

Description

**Is your feature request related to a problem? Please describe.**
Currently, when building vocabulary, user can only say need_pad, then Vocabulary assign a fix index to ['PAD'], which is 0 or -1 (if 'one-hot'). However, in some cases, user may want to use a specific index for ['PAD'].

**Describe the solution you'd like**
Make pad_value and unk_value configurable.

**Describe alternatives you've considered**
A clear and concise description of any alternative solutions or features you've considered.

**Additional context**
Add any other context or screenshots about the feature request here.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Vocabulary implementation and its existing handling of need_pad, the PAD index, one-hot mode, and UNK. Review related tests if present, then define the configurable pad_value and unk_value behavior and verify that callers can assign specific indices without breaking current defaults.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.