Error Training LGBM Classifier on Databricks

Nobody has claimed this yet.

HI,
When I tried to train a classification model using LightGBM in a Databricks Serverless environment, I encountered the following error:
['str' object has no attribute 'getParam']

I cannot confirm whether it's an environment issue or a package issue.

However, I am currently attempting to pinpoint the potential environment-related problems.

If anyone has any insights, I would greatly appreciate your guidance.

**environment**
Databricks Serverless 4
pyspark : 4.0.0

**code**

`model = LightGBMClassifier(
objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True
)

model = model.fit(trin_date)`

**Error**

`AttributeError: 'str' object has no attribute 'getParam'
[Trace ID: 00-678e75eae65d2f9e8bc744eb078227e0-eb6e67874598c4bf-00]
File , line 33
27 # trin_date.display()
29 model = LightGBMClassifier(
30 objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True
31 )
---> 33 model = model.fit(trin_date)
File , line 33
27 # trin_date.display()
29 model = LightGBMClassifier(
30 objective="binary", featuresCol="features", labelCol="buy_flg", isUnbalance=True
31 )
---> 33 model = model.fit(trin_date)
File /databricks/python_shell/lib/dbruntime/MLWorkloadsInstrumentation/_pyspark.py:30, in _create_patch_function..patched_method(self, *args, **kwargs)
28 call_succeeded = False
29 try:
---> 30 result = original_method(self, *args, **kwargs)
31 call_succeeded = True
32 return result
File /databricks/python/lib/python3.12/site-packages/pyspark/ml/base.py:203, in Estimator.fit(self, dataset, params)
201 return self.copy(params)._fit(dataset)
202 else:
--> 203 return self._fit(dataset)
204 else:
205 raise TypeError(
206 "Params must be either a param map or a list/tuple of param maps, "
207 "but got %s." % type(params)
208 )
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/lightgbm/LightGBMClassifier.py:2148, in LightGBMClassifier._fit(self, dataset)
2147 def _fit(self, dataset):
-> 2148 java_model = self._fit_java(dataset)
2149 return self._create_model(java_model)
File /databricks/python/lib/python3.12/site-packages/pyspark/ml/wrapper.py:412, in JavaEstimator._fit_java(self, dataset)
397 """
398 Fits a Java model to the input dataset.
399
(...)
408 fitted Java model
409 """
410 assert self._java_obj is not None
--> 412 self._transfer_params_to_java()
413 return self._java_obj.fit(dataset._jdf)
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/core/schema/Utils.py:134, in ComplexParamsMixin._transfer_params_to_java(self)
132 self._java_obj.set(pair)
133 if self.hasDefault(param):
--> 134 pair = self._make_java_param_pair(param, self._defaultParamMap[param])
135 pair_defaults.append(pair)
136 if len(pair_defaults) > 0:
File /local_disk0/.ephemeral_nfs/envs/pythonEnv-b17e7c41-2947-4628-8639-a4f59e8d5edb/lib/python3.12/site-packages/synapse/ml/core/serialize/java_params_patch.py:122, in _mml_make_java_param_pair(self, param, value)
120 sc = SparkContext._active_spark_context
121 param = self._resolveParam(param)
--> 122 java_param = self._java_obj.getParam(param.name)
123 java_value = _mml_py2java(sc, value)
124 return java_param.w(java_value)`

Start by reviewing the LightGBMClassifier._fit path and the PySpark parameter transfer frames shown in the report, then reproduce the failure with Databricks Serverless 4, PySpark 4.0.0, and the reported SynapseML setup. Done means identifying whether the incompatibility is in the package or environment and documenting or correcting the supported behavior.

Written by the indexing model from the issue text.