Connecting Databricks Delta Lake using SQL Alchemy (Pyhive)
- Dominant language
- Python
- Stars
- 1.7k
- Forks
- 545
- PR merge metrics
- No merged PRs in 30d
Description
1. I have used the Flask and SQL Alchemy ORM concept for the REST API.
2. The API simply connects to one of the Databricks Delta lake database , queries one particular table and returns the result.
3. I have used the following sample code to create the SQL Alchemy engine : URL: https://pypi.org/project/databricks-dbapi/
_Azure Databricks with token
provide token, region for url, database name, http_path (with cluster name)
engine = create_engine(
"databricks+pyhive://token:@.azuredatabricks.net:443/",
connect_args={"http_path": ""}
)_
4. I have written the REST API , created a docker image and deployed the image on the AKS cluster.
5. For starting few days the API was fetching the response. From yesterday I see the attached whole error received DatabricksI_Error.txt)
Single line from the error : Error Line: sqlState=None, errorCode=0, errorMessage='Invalid SessionHandle: SessionHandle [4d00c5b7-2ee4-448a-92de-cb04deb773b6]'), operationHandle=None)
6. I had restarted the docker container yesterday which has this POD using the agent-pool-node serial console after which the API was working. But from today morning again I get this issue. I m not sure why the issue is occurring could you please help me ?
[Databricks_Error.txt](https://github.com/dropbox/PyHive/files/3818744/Databricks_Error.txt)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.