--env-file does not support inner quotes (does not behave like dotenv)

Version

v20.16.0

Platform

Darwin <redacted> 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:14:38 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6020 arm64

Subsystem

No response

What steps will reproduce the bug?

inspired by this comment: https://github.com/nodejs/node/pull/50814#issuecomment-1817949911

1. create an .env file:

   INNER_QUOTES="1: foo'bar"baz`qux"
   INNER_QUOTES_WITH_NEWLINE="2: foo bar\ni am "on" newline, 'yo'"
   

2. test with dotenv:

   // envtest-dotenv.js
   require("dotenv").config();
   console.log(process.env.INNER_QUOTES);
   console.log(process.env.INNER_QUOTES_WITH_NEWLINE);
   

   $ node envtest-dotenv.js

3. test with --env-file

   // envtest.js
   console.log(process.envconsole.log(process.env.INNER_QUOTES);
   console.log(process.env.INNER_QUOTES_WITH_NEWLINE);
   

   $ node --env-file=.env envtest.js

How often does it reproduce? Is there a required condition?

always

What is the expected behavior? Why is that the expected behavior?

outputs should be the same.

// dotenv output:

1: foo'bar"baz`qux
2: foo bar
i am "on" newline, 'yo'

What do you see instead?

output are different, node native terminates at the first occurrence of the double quote:

// --env-file output
1: foo'bar
2: foo bar
i am

compare that to dotenv:

1: foo'bar"baz`qux
2: foo bar
i am "on" newline, 'yo'

Additional information

• this makes it impossible to have env-variables that contain all three quotes (", ' and `) and newlines.
• There is no alternative to inner quotes, because escaping quotes have been rejected https://github.com/nodejs/node/pull/50814
• It's very problematic in many cases since depending on the quotes you are using, you need to use different outer quotes
• i noticed that because i am writing some tooling that creates those .env files based on some other configuration. I just used double quotes as outer quotes and noticed that it fails when the string contains double quotes
• be aware that dotenv also has a bug, when your string contains an actual newline (not "\n") AND has inner quotes, it will cut the string off at the line break. It works if the string does not contain inner quotes. It also works if you instead use "\n" instead of an actual line break. The output will still be the same and will have a real line break in it (as long as you put outer quotes around the string. without quotes, the \n will be literally in the output as "\n")

And

• since the format is so problematic, maybe its time for a better format?

Start by reproducing the .env examples with node --env-file=.env envtest.js and compare the output with dotenv. Trace the --env-file parsing entry point and existing tests, then make the native parser preserve inner quotes and newlines consistently with the expected output and add coverage for these cases.