repo.Network.ListReferences(remote) throwing exception "this remote has never connected" intermittently

Calling repo.Network.ListReferences(remote) throws exception "this remote has never connected" intermittently. This is how I have called it ... if it matters

          string localRepoPath = "path to cloned folder";
          string remoteName = "origin";            
            using (Repository repo = new Repository(localRepoPath))
            {
                remoteName = repo.Network.Remotes.FirstOrDefault()?.Name ?? remoteName;
                Remote remote = repo.Network.Remotes[remoteName];                
                List<Reference> references = repo.Network.ListReferences(remote).ToList();
				
				// some more code logic here
			}

So am calling the overload of ListReferences(Remote remote) which essentially calls ListReferencesInternal(string url, CredentialsHandler credentialsProvider) passing null to CredentialsHandler. Should this be an issue? Don't think so since it's working in second try or so. Relevant exception stack trace below

{
"Depth": 1,
"ClassName": "",
"Message": "this remote has never connected",
"Source": "LibGit2Sharp",
"StackTraceString": " at LibGit2Sharp.Core.Ensure.HandleError(Int32 result)\r\n at LibGit2Sharp.Core.Proxy.git_remote_ls(Repository repository, RemoteHandle remote)\r\n at LibGit2Sharp.Network.ListReferencesInternal(String url, CredentialsHandler credentialsProvider)\r\n

Please suggest, what's wrong is going on (Or) if you need further information from my end.

Found this existing issue https://github.com/libgit2/libgit2sharp/issues/1504 but am not passing any credential, and it's a public repo which am trying to access.

Start with Network.ListReferences(Remote), ListReferencesInternal, and the git_remote_ls call shown in the stack trace. Reproduce the first and subsequent calls using the provided sample, then trace the intermittent remote state; done means the cause is identified and the failure has a regression test or a clearly documented resolution.