Custom server config not working

Link to the code that reproduces this issue

https://codesandbox.io/p/devbox/next-js-custom-server-bug-6z8fhc

To Reproduce

1. Create a basic next js application by running npx create-next-app@latest.
2. cd into the new directory and add a basic server.js file in the root directory with these content:

const { createServer } = require("http");
const { parse } = require("url");
const fs = require("fs");
const path = require("path");

const next = require("next");

const dev = process.env.NODE_ENV !== "production";
const hostname = "localhost";
const port = 3600;

const app = next({
    dev,
    hostname,
    port,
    conf: {
        distDir: ".custom_dist",
    }
});

const handle = app.getRequestHandler();

app.prepare()
    .then(() => {
        const server = createServer(async (req, res) => {
            try {
                const parsedUrl = parse(req.url || "", true);
                const { pathname, query } = parsedUrl;

                await handle(req, res, parsedUrl);
            } catch (err) {
                console.error("Error occurred handling", req.url, err);
                res.statusCode = 500;
                res.end("internal server error");
            }
        });

        server.on("error", (err) => {
            console.error("⚠️ SERVER ERROR =>", err.message);
        });

        server.listen(port, () => {
            console.log(`> Ready on http://${hostname}:${port}`);
        });
    })
    .catch((error) => {
        console.log(error);
    });

3. Run node server.js to start the development server.

Current vs. Expected behavior

Expectation

When the development server is started, the dist directory should be loaded from the conf property in the next function argument, and there should be a new folder in the root directory named .custom_dist.

Current Behavior

The default .next directory is used instead of the specified .custom_dist directory.

Verify canary release

• I verified that the issue exists in the latest Next.js canary release

Provide environment information

Operating System:
  Platform: linux
  Arch: x64
  Version: #1 SMP PREEMPT_DYNAMIC Sun Aug  6 20:05:33 UTC 2023
Binaries:
  Node: 20.9.0
  npm: 9.8.1
  Yarn: 1.22.19
  pnpm: 8.10.2
Relevant Packages:
  next: 14.0.4
  eslint-config-next: 14.0.4
  react: 18.2.0
  react-dom: 18.2.0
  typescript: 5.3.3
Next.js Config:
  output: N/A

Which area(s) are affected? (Select all that apply)

Not sure

Additional context

This bug appeared with Next 13. I'm currently running Next 12 on my apps to avoid this bug.

Start by reproducing the behavior with the linked CodeSandbox or the shown server.js and run node server.js after creating a basic Next.js app. Trace how the conf option passed to next() is handled in the custom-server path. Done means development uses .custom_dist instead of .next, with the behavior verified against the reproduction steps.