microsoft / microsoft/SynapseML

[BUG]An error occurred while calling o1156.fit. : java.lang.UnsatisfiedLinkError:

Open
#1,811 21 comments 0 reactions 2 assignees Claimed by @imatiach-msft View on GitHub
area/lightgbm bug
Dominant language
Scala
Stars
5.2k
Forks
868
Avg merge
22h 9m
Merged PRs (30d)
45

Description

### SynapseML version

0.10.2

### System information

- **Language version** : python 3.9
- **Spark Version** : 3.2.1
- **Spark Platform** : EMR Notebook (Pyspark Kernel)

### Describe the problem

I'm testing a demo code for using **LightGBMClassifier** on both local machine (with Spark 3.3)and on EMR Cluster.
And I am stuck at the same following error.

From EMR Notebook
Screen Shot 2023-01-28 at 10 01 55 PM

From Local Machine
Screen Shot 2023-01-28 at 10 01 01 PM

### Code to reproduce issue

..........
feature_cols = train_df.columns[1:]
featurizer = VectorAssembler(inputCols=feature_cols, outputCol='features')
train_data = featurizer.transform(train_df)
model = LightGBMClassifier(objective="binary", featuresCol="features", labelCol="is_useful", isUnbalance=True)
model_train = model.fit(train_data)

### Other info / logs

_No response_

### What component(s) does this bug affect?

- [ ] `area/cognitive`: Cognitive project
- [ ] `area/core`: Core project
- [ ] `area/deep-learning`: DeepLearning project
- [X] `area/lightgbm`: Lightgbm project
- [ ] `area/opencv`: Opencv project
- [ ] `area/vw`: VW project
- [ ] `area/website`: Website
- [ ] `area/build`: Project build system
- [ ] `area/notebooks`: Samples under notebooks folder
- [ ] `area/docker`: Docker usage
- [ ] `area/models`: models related issue

### What language(s) does this bug affect?

- [ ] `language/scala`: Scala source code
- [X] `language/python`: Pyspark APIs
- [ ] `language/r`: R APIs
- [ ] `language/csharp`: .NET APIs
- [ ] `language/new`: Proposals for new client languages

### What integration(s) does this bug affect?

- [X] `integrations/synapse`: Azure Synapse integrations
- [ ] `integrations/azureml`: Azure ML integrations
- [ ] `integrations/databricks`: Databricks integrations

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.