SNOW-862000: Support for External Authentication while running from EMR using spark-magic

What is the current behavior?

Currently while trying to use the snowflake.connector.connect function, and setting the authenticator to externalbrowser, the process works by starting a socket server, and giving an sso url that needs to be opened in an external browser to authenticate.

While this authentication happens, the process is waiting for input on the line
https://github.com/snowflakedb/snowflake-connector-python/blob/7ac23578fa906cf9e969a6fff2ed4a202a7b69b0/src/snowflake/connector/auth/webbrowser.py#L167

This method of waiting for user input doesn't seem to work when we are trying to authenticate from a jupyter-notebook that is connected to an EMR through spark-magic. In this example, I tried it from AWS sagemaker.

This doesn't work with spark-magic because spark-magic doesn't support waiting for user input. Hence this call ends up throwing the following error:

An error was encountered:
EOF when reading a line
Traceback (most recent call last):
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/__init__.py", line 51, in Connect
    return SnowflakeConnection(**kwargs)
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 319, in __init__
    self.connect(**kwargs)
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 590, in connect
    self.__open_connection()
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 860, in __open_connection
    self.authenticate_with_retry(self.auth_class)
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 1127, in authenticate_with_retry
    self._authenticate(auth_instance)
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 1145, in _authenticate
    password=self._password,
  File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/auth/webbrowser.py", line 167, in prepare
    url = input("Enter the URL the SSO URL redirected you to: ")
EOFError: EOF when reading a line

One workaround is to read the data locally using the %%local magic command from spark-magic, and sending it to the spark cluster. But that is inefficient since it is not scalable for large datasources.

What is the desired behavior?

One suggestion that I have to solve this issue is, instead of waiting for user input directly, can we have another function to feed the redirect url.
Hence we can get the SSO url in one function call. That function will return the sso url, while the socket server is kept running.

A second function can be called once we have the redirect url, which then posts it to the socket server, completing the authentication process.

This workaround should help add support for spark-magic.

This is one of the ideas that I can think of to circumvent the limitation posed by spark-magic. If there is a much cleaner way to get this done, suggestions are welcome.

How would this improve snowflake-connector-python?

This would add support for spark-magic, and allow users to connecting to Spark clusters remotely to use this library and read from snowflake directly from inside the cluster.

References and other background

No response

Start with src/snowflake/connector/auth/webbrowser.py at line 167, then follow the authentication calls in connection.py shown in the traceback. Check how the socket server and redirect URL are currently handled. Done should mean external authentication can receive the redirect URL without waiting for spark-magic to provide stdin.