useSession only getting the session after manually reloading the page

Environment

System:
OS: Linux 5.15 Ubuntu 22.04.3 LTS 22.04.3 LTS (Jammy Jellyfish)
CPU: (6) x64 Common KVM processor
Memory: 48.93 GB / 62.79 GB
Container: Yes
Shell: 5.1.16 - /bin/bash
Binaries:
Node: 20.10.0 - ~/.nvm/versions/node/v20.10.0/bin/node
npm: 10.2.3 - ~/.nvm/versions/node/v20.10.0/bin/npm
npmPackages:
@auth/core: 0.18.1 => 0.18.1
next: 14.0.2 => 14.0.2
next-auth: ^5.0.0-beta.4 => 5.0.0-beta.4
react: ^18 => 18.2.0
Browser:
Chrome
Fireox

Reproduction URL

https://github.com/getr62/discuss

Describe the issue

I am using the credential provider, because I have to develop an app which is only available in a local network, maybe later also running on computers even without connection to the internet, so OAuth etc is no option for me.

src/lib/auth.ts

import NextAuth from 'next-auth';
import CredentialsProvider from 'next-auth/providers/credentials';
import { z } from 'zod';
import { db } from '@/db';
import type { User } from '@prisma/client';

async function getUser(name: string): Promise<User | null> {
  try {
    const user = await db.user.findFirst({
      where: {
        name,
      },
    });
    return user;
  } catch (error) {
    throw new Error('Failed to fetch user.');
  }
}

export const {
  handlers: { GET, POST },
  auth,
  signOut,
  signIn,
} = NextAuth({
  providers: [
    CredentialsProvider({
      name: 'Credentials',
      async authorize(credentials) {
        const parsedCredentials = z
          .object({
            username: z.string(),
            password: z.string(),
          })
          .safeParse(credentials);

        // console.log('credentials: ', credentials);
        if (parsedCredentials.success) {
          const { username, password } = parsedCredentials.data;
          const user = await getUser(username);
          console.log('USER in auth.ts AUTHORIZE: ', user);
          if (!user) return null;
          return user;
        }

        return null;
      },
    }),
  ],
  session: {
    strategy: 'jwt',
  },
  secret: process.env.AUTH_SECRET,
  callbacks: {
    async jwt({ token, user }: any) {
      console.log('USER in JWT callback: ', user);
      console.log('TOKEN BEFORE modification in JWT callback: ', token);
      if (token && user) {
        token.role = user.role;
        token.id = user.id;
      }
      console.log('TOKEN AFTER modification in JWT callback: ', token);
      return token;
    },
    async session({ session, token, user }: any) {
      console.log('USER in SESSION callback: ', user);
      console.log('TOKEN BEFORE modification in SESSION callback: ', token);
      console.log('SESSION BEFORE modification in SESSION callback: ', session);
      if (session && token) {
        session.user.id = token.id;
        session.user.role = token.role;

        console.log('SESSION AFTER modification in SESSION callback: ', session);
      }

      return session;
    },
  },
  pages: {
    signIn: '/src/components/auth/sign-in-form',
  },
});

In Next.js I am using the app router with server actions, also for the sign-in functionality.

src/actions/sign-in.tx

'use server';

import type { Profile } from '@prisma/client';
import { z } from 'zod';
import { db } from '@/db';
import { redirect } from 'next/navigation';
import paths from '@/lib/paths';
import * as auth from '@/lib/auth';
import { compare } from 'bcrypt';

const signInSchema = z.object({
  username: z.string().min(3),
  password: z.string().min(7),
});

interface SignInFormState {
  errors: {
    username?: string[];
    password?: string[];
    _form?: string[];
  };
}

export async function signIn(
  formState: SignInFormState,
  formData: FormData
): Promise<SignInFormState> {
  const result = signInSchema.safeParse({
    username: formData.get('username'),
    password: formData.get('password'),
  });

  console.log('result signInSchema safeParse: ', result);

  if (!result.success) {
    return {
      errors: result.error.flatten().fieldErrors,
    };
  }

  const user = await db.user.findUnique({
    where: {
      name: result.data.username,
    },
  });

  if (!user) {
    return {
      errors: {
        _form: ['Wrong credentials'],
      },
    };
  }

  const passwordsMatch = await compare(result.data.password, user.password);

  if (!passwordsMatch) {
    return {
      errors: {
        _form: ['Wrong credentials'],
      },
    };
  }

  console.log('user found in sign-in action: ', user);

  let profile: Profile;
  try {
    await auth.signIn('credentials', {
      redirect: false,
      username: result.data.username,
      password: result.data.password,
    });

    profile = await db.profile.upsert({
      where: {
        userId: user.id,
      },
      update: {
        lastLogin: new Date().toLocaleString(),
      },
      create: {
        userId: user.id,
        lastLogin: new Date().toLocaleString(),
      },
    });
  } catch (err: unknown) {
    if (err instanceof Error) {
      console.log('error in auth.signIn action: ', err);
      return {
        errors: {
          _form: [err.message],
        },
      };
    } else {
      return {
        errors: {
          _form: ['Something went wrong'],
        },
      };
    }
  }

  redirect(paths.home());
  return {
    errors: {},
  };
}

The logged in user should be displayed in the header component. The header is part of the root-layout file. To avoid that in the build process every page is treated as a dynamic route, this on the authentication state depending part is put in a client component.

src/components/header

import Link from 'next/link';
import { Suspense } from 'react';
import {
  Navbar,
  NavbarBrand,
  NavbarContent,
  NavbarItem,
} from '@nextui-org/react';
import HeaderAuth from '@/components/header-auth';
import SearchInput from '@/components/search-input';
import dynamic from 'next/dynamic';

const DynamicHeaderAuth = dynamic(() => import('./header-auth'), {
  ssr: false,
});

export default function Header() {
  return (
    <Navbar className='shadow mb-6'>
      <NavbarBrand>
        <Link href='/' className='font-bold'>
          Discuss
        </Link>
      </NavbarBrand>
      <NavbarContent justify='center'>
        <NavbarItem>
          <Suspense>
            <SearchInput />
          </Suspense>
        </NavbarItem>
      </NavbarContent>

      <NavbarContent justify='end'>
        {/* <HeaderAuth /> */}
        <DynamicHeaderAuth />
      </NavbarContent>
    </Navbar>
  );
}

src/components/header-auth

'use client';

import Link from 'next/link';
import {
  Chip,
  NavbarItem,
  Button,
  Popover,
  PopoverTrigger,
  PopoverContent,
} from '@nextui-org/react';
import { Icon } from 'react-icons-kit';
import { pacman } from 'react-icons-kit/icomoon/pacman';
import { useSession } from 'next-auth/react';
import * as actions from '@/actions';

export default function HeaderAuth() {
  const session = useSession();
  console.log('session 1 from useSession in header-auth: ', session);

  let authContent: React.ReactNode;
  if (session.status === 'loading') {
    authContent = null;
  } else if (session.data?.user) {
    console.log('session 2 from useSession in header-auth: ', session);
    authContent = (
      <Popover placement='left'>
        <PopoverTrigger>
          <Chip
            className='cursor-pointer'
            startContent={<Icon icon={pacman} />}
            variant='faded'
            color='default'
          >
            {session.data.user.name}
          </Chip>
        </PopoverTrigger>
        <PopoverContent>
          <div className='p-4'>
            <form action={actions.signOut}>
              <Button type='submit'>Sign Out</Button>
            </form>
          </div>
        </PopoverContent>
      </Popover>
    );
  } else {
    authContent = (
      <>
        <NavbarItem>
          <Link href={'/sign-in'}>Sign In</Link>
        </NavbarItem>

        <NavbarItem>
          <Link href={'/sign-up'}>Sign Up</Link>
        </NavbarItem>
      </>
    );
  }

  return authContent;
}

As you can see in the auth.ts file I am heavily logging different stages in the jwt and session. Basically I am able to prep the token with the user id and role as well as all these values into the session with just one big problem. The session is only created, after I manually reload the page. To showcase I uploaded a video where you can see in the browser console that the session state only changes from unauthenticated to authenticated when the page is manually reloaded.

https://github.com/nextauthjs/next-auth/assets/25088567/a814a6bb-cecf-43ae-9a50-3f048f895c1a

Sure, I am by far no experienced developer, but I read in the discussions that some other people have issues too using the useSession hook and not getting the session back from the server.

How to reproduce

I linked a small example project, where I was experimenting with these features like server actions. In there should be everything to run the app with the exception of the .env file which contains these values:

DATABASE_URL="postgresql://user:password@localhost:5434/discuss_ext?schema=public"
AUTH_SECRET=************
NEXTAUTH_URL=http://localhost:3004

As you can see, nextjs runs on port 3004 which is hard coded in project.json and the database port is set to port 5434, which is also used in the docker-compose.yml

Expected behavior

After signing into the app the authentication state should change automatically from unauthenticated to authenticated.

Start with the linked reproduction and trace the sign-in flow through src/actions/sign-in.tx, src/lib/auth.ts, and src/components/header-auth, focusing on the useSession state after the server action redirects. Compare the JWT and session callback logs with the client component's status transition. Done means the header changes from unauthenticated to authenticated after signing in without a manual reload.