test.txt problem

I am training a model using the Kaggle dataset. The Kaggle dataset consists of train.txt and test.txt files.

The train.txt used during training is well preprocessed, and the training is completed successfully.

However, it seems that the dataset test.txt used for inference is not being properly preprocessed.

"What seems to be the problem?"

python data_utils.py --raw-data-file=/data/janghobaek/test.txt

Start by reading data_utils.py and reproducing the command with /data/janghobaek/test.txt. Compare how the Kaggle train.txt and test.txt inputs are processed; the issue is complete when the preprocessing discrepancy is identified and the expected inference behavior is clearly established.

Written by the indexing model from the issue text.