Can you write some docs or notes to explain the content of the data pre-processing，such as fucn: _process_list ?
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 176
- PR merge metrics
- No merged PRs in 30d
Description
数据预处理的步骤似乎有一点复杂。
我阅读了一段时间的代码之后，还是不是很明白有些预处理的代码目的是做什么？
作者能否写个文档或者注释，阐述一下一些数据预处理相关函数的具体左右，比如convert_data_to_id, _convert_to_tensors ？
有些变量的含义也不是太理解，比如 input_id_subs,ext_tables,segment_rel，segment_rel_offset 🌝
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating the data-preprocessing implementations for convert_data_to_id, _convert_to_tensors, and _process_list. Trace how input_id_subs, ext_tables, segment_rel, and segment_rel_offset are used, then document the processing steps and the purpose of these functions and variables; done means a newcomer can follow the preprocessing flow from the documentation or comments.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- documentation, machine-learning
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100