Reproduction Issue of BiT on STS-B

Great work and thanks a lot for opening up the great work!

While reusing the code released, I found some issues below:

I can not reproduce the W1A1 version BiT accuracy on the STS-B dataset as reported in the paper (68.7% vs 71.1%).

I have basically followed the setting in the code and paper, can you share some suggestions for the issue?
https://github.com/facebookresearch/bit/blob/37d2bd73111dda9787424bc9e3a48edb7f18cc88/utils_glue.py#L689