Method public org.apache.spark.sql.streaming.DataStreamReader org.apache.spark.sql.SQLContext.readStream() is not whitelisted on class class org.apache.spark.sql.SQLContext

Hello I'm having problems reading in streams using evenhubs inside azure databricks, the read works using an interactive cluster but when i try to use the same code inside of a high currency cluster with the role access control enabled (only accepts sql and python code),i got the following error:

py4j.security.Py4JSecurityException: Method public org.apache.spark.sql.streaming.DataStreamReader org.apache.spark.sql.SQLContext.readStream() is not whitelisted on class class org.apache.spark.sql.SQLContext

Featured used :
Runtime 6.4(scala 2.11, Spark 2.4.5)

Spark configuration:
spark.databricks.cluster.profile serverless
spark.databricks.io.cache.maxMetaDataCache 1g
spark.databricks.io.cache.maxDiskUsage 80g
spark.databricks.acl.dfAclsEnabled true
spark.databricks.delta.preview.enabled true
spark.databricks.io.cache.compression.enabled false
spark.databricks.repl.allowedLanguages python,sql

Library installed :
com.microsoft.azure:azure-eventhubs-spark_2.11:2.3.6

No repository file or test is identified. Start by reproducing the Event Hubs readStream call on Runtime 6.4 in an Azure Databricks high-concurrency cluster with the listed ACL and language settings, then compare it with interactive-cluster behavior; done means documenting the confirmed cause and supported resolution or limitation.

由索引模型根据 Issue 内容生成。