Authentication eventually fails when locking dependencies takes too long in poetry

We have a python library published in an Artifact Registry repository. When adding it as a source to poetry, it always checks this index for every single dependency. Since most dependencies are only hosted on PyPI, poetry keeps getting 404 responses from Artifact Registry which is fine. However, it also randomly gets 401s, as if it were not authenticating properly. The problem is that when locking takes too long, e.g. more than an hour, eventually it starts getting only 401s. Therefore, when it checks for our private package, it eventually doesn't find it either, hence failing completely.

Do you have any suggestions as to where the problem could be? Please let me know if you need any further details.