hankcs / hankcs/ID-CNN-CWS

Results of the experiment cannot be reproduced on PKU, trainning stopped after 1 iteration

Open
#6 2 comments 0 reactions 0 assignees View on GitHub
duplicate
Dominant language
Python
Stars
132
Forks
37
PR merge metrics
No merged PRs in 30d

Description

我们的实验环境 : Python 3.5 Tensorflow 1.4.0 4x Nvidia Geforce 1080Ti
对代码进行了小幅调整以便在我们的环境上运行
第一个实验测试的是Sighan2005-PKU
但是,无论BI-LSTM-CRF还是ID-CNN-CRF 每次训练完第1个迭代后,验证集、测试集都只有0.58-0.62 然后提示 'Score too low, break to save time' 退出。
同时,用同样脚本分割的语料训练 foolNLTK(https://github.com/rockyzhengwu/FoolNLTK), 几乎一样的BI-LSTM-CRF第一轮训练后验证集有0.88的F1值。
初步怀疑评测脚本有问题,验证集、测试集分数一致

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reproducing the PKU experiment with Python 3.5 and TensorFlow 1.4.0, using the same corpus-splitting and training scripts described in the issue. Inspect the evaluation path around the “Score too low, break to save time” message and compare validation and test results with the reported FoolNLTK baseline. Done means identifying whether evaluation is incorrect or training stops prematurely and documenting a verified correction.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.