Callback url not set correctly if using custom login page

Environment

System:
OS: macOS 12.6
CPU: (10) arm64 Apple M1 Pro
Binaries:
Node: 16.17.0 - ~/.nvm/versions/node/v16.17.0/bin/node
Yarn: 1.22.15 - ~/.nvm/versions/node/v16.17.0/bin/yarn
npm: 8.15.0 - ~/.nvm/versions/node/v16.17.0/bin/npm
Browsers:
Firefox: 104.0.2
npmPackages:
next: 12.3.0 => 12.3.0
next-auth: 4.10.3 => 4.10.3
react: 18.2.0 => 18.2.0

Reproduction URL

https://github.com/voinik/next-auth-wrong-callbackurl

Describe the issue

If you have a custom login page and try to access a page that requires you to be authenticated, then you get redirected to that login page, which is correct. You also get a redirectUrl=... added to the url during the redirect to your login page, which is also correct.

But when you subsequently login through email (I haven't tried other Providers), the url that gets sent to the email address contains an incorrect callbackUrl.

Let's say we tried to go to the /client route but were redirected to the login page because the /client page requires authentication. And let's say we login through email. The url we expect to get in the email is something like localhost:3000/api/auth/callback/email?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Fclient&token=..... or similar. But what we actually get is: http://localhost:3000/api/auth/callback/email?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Flogin%3FcallbackUrl%3D%252Fclient&token=......

Notice how the callbackUrl contains another callbackUrl.

Edit: If we go to the homepage and click on the "Log in" button I created on the far right of the header, then the callbackUrl we eventually get is: http://localhost:3000/api/auth/callback/email?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Flogin%3FcallbackUrl%3Dhttp%253A%252F%252Flocalhost%253A3000%252F&token=...... Notice how the complete baseUrl is inside the nested callbackUrl. To fix this I added the ternary check in the first return in the circumvention code below.

I managed to circumvent the issue by adding this to my callbacks object in my options object:

async redirect({ baseUrl, url }) {
    const redirectUrl = decodeURIComponent(url);
    const callbackIndex = redirectUrl.indexOf('callbackUrl=');
    if (callbackIndex > -1) {
        const callbackPath = redirectUrl.slice(callbackIndex + 12);
        // If I try to login from my homepage, the nested callbackUrl contains the full baseUrl.
        // This behavior seems to be triggerd if you call `signIn()` from a page.
        return callbackPath.includes(baseUrl) ? callbackPath : baseUrl + callbackPath;
    }
    return url;
},

But this obviously not a great long term solution.

How to reproduce

Here's a copy of the README of my repo:

How to reproduce the bug

1. Add an .env file with NEXTAUTH_SECRET=heheheh
2. Run npm install
3. Start the project
4. Go to http://localhost:3000/
5. I've setup the middleware to require the user to be authenticated when going to the /client route, so click on the Client anchor tag
6. You'll end up on the custom login page I created. Click on Do it! to attempt to log in.
7. First of all, notice the url bar. It should say http://localhost:3000/login?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Flogin%3FcallbackUrl%3D%252Fclient&error=EmailSignin. Notice how the callbackUrl refers to the url of the custom login page that itself has a callbackUrl.
8. Now check your terminal. I've logged the url that gets sent to the sendVerificationRequest function, which is the url that gets sent in the login email. It should look like this: url: http://localhost:3000/api/auth/callback/email?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Flogin%3FcallbackUrl%3D%252Fclient&token=.......

Notice how the callbackUrl that ends up in the email is the base url with a callback url that contains the url of the login page that itself again has a callbackUrl.

This means that when a user gets redirected to the login page (because they're attempting to access a page that requires them to be logged in), they get redirected to... the login page itself, rather than where they came from. The user will arrive on the login page and there will be a callbackUrl in the url bar to the page they actually want to go to.

Expected behavior

When using custom login pages we expect the url in the email to be something like localhost:3000/api/auth/callback/email?callbackUrl=http%3A%2F%2Flocalhost%3A3000%2Fclient&token=..... or similar. Instead we're getting a callbackUrl inside another callbackUrl.

Run the linked reproduction steps, then inspect the custom login page, middleware protecting /client, and the sendVerificationRequest URL logging. Trace how the redirect callback receives and propagates callbackUrl; done means the email URL points directly to the originally requested page without nested callback URLs.