Custom OAuth provider and 2FA: no TOTP popup

Description:

I have a custom OAuth provider set up with RocketChat, which is working fine. When a user enables 2FA, however, things start breaking.

Steps to reproduce:

1. Have a RC instance with custom OAuth
2. enable 2FA with TOTP for a single user
3. log out as that user and try to relogin

Expected behavior:

A user gets asked for their 2FA TOTP token, in a reliable fashion. And there is some JS API to detect in the browser that this request is open.

Actual behavior:

There is no 2FA popup on the client 99% of the time; I sometimes see it, but I have no reliable way of reproducing it.

There is a server-side error logged:

Exception while invoking method login errorClass [Error]: TOTP Required [totp-required]
    at checkCodeForUser (app/2fa/server/code/index.ts:194:9)
    at app/2fa/server/loginHandler.ts:48:3
    at Callbacks.runOne (lib/callbacks.ts:263:11)
    at lib/callbacks.ts:275:17
    at lib/callbacks.ts:283:5
    at lib/callbacks.ts:291:12
    at Callbacks.run (lib/callbacks.ts:403:10)
    at app/authentication/server/startup/index.js:362:20
    at packages/callback-hook/hook.js:141:18
    at packages/accounts-base/accounts_server.js:225:15
    at Hook.forEach (packages/callback-hook/hook.js:110:15)
    at AccountsServer._validateLogin (packages/accounts-base/accounts_server.js:222:29)
    at AccountsServer._attemptLogin (packages/accounts-base/accounts_server.js:472:10)
    at MethodInvocation.methods.login (packages/accounts-base/accounts_server.js:654:23)
    at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1885:12)
    at packages/ddp-server/livedata_server.js:1803:15
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at packages/ddp-server/livedata_server.js:1801:36
    at new Promise (<anonymous>)
    at Server.applyAsync (packages/ddp-server/livedata_server.js:1800:12)
    at Server.apply (packages/ddp-server/livedata_server.js:1739:26)
    at Server.call (packages/ddp-server/livedata_server.js:1721:17)
 => awaited here:
    at Promise.await (/app/bundle/programs/server/npm/node_modules/meteor/promise/node_modules/meteor-promise/promise_se

    at Server.apply (packages/ddp-server/livedata_server.js:1752:22)
    at Server.call (packages/ddp-server/livedata_server.js:1721:17)
    at Object.post (app/api/server/v1/misc.ts:612:27)
    at app/api/server/api.js:463:96
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1257:12)
    at Object._internalRouteActionHandler [as action] (app/api/server/api.js:463:39)
    at Route._callEndpoint (packages/rocketchat_restivus/lib/route.coffee:150:32)
    at packages/rocketchat_restivus/lib/route.coffee:59:33
    at packages/simple_json-routes.js:100:9 {
  isClientSafe: true,
  error: 'totp-required',
  reason: 'TOTP Required',
  details: {
    method: 'totp',
    codeGenerated: false,
    availableMethods: [ 'totp' ]
  },
  errorType: 'Meteor.Error'

Server Setup Information:

• Version of Rocket.Chat Server: 5.4.10
• Operating System: (official RocketChat Docker container)
• Deployment Method: Docker for local development
• Number of Running Instances: 1
• DB Replicaset Oplog: ???
• NodeJS Version: 14.19.3
• MongoDB Version: 5.0.18

Client Setup Information

• Desktop App or Browser Version: Firefox 115, Chromium 115
• Operating System: Ubuntu 20.04

Additional context

Relevant logs:

Start with app/2fa/server/code/index.ts and app/2fa/server/loginHandler.ts, then trace the login flow through app/authentication/server/startup/index.js and the client-side OAuth handling. Reproduce a custom OAuth login with TOTP enabled in the listed browser environments. Done means the TOTP prompt appears reliably and the browser can detect that the request is open.