-
Notifications
You must be signed in to change notification settings - Fork 833
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Error while using "LightGMB" on Fabric #2242
Comments
Please refer to microsoft/LightGBM#6492 which was initially raised with LightGBM initially. |
Tagging on to this; I am seeing the same error in the same Fabric Spark runtime. |
I've got the same issue when running the following script on Synapse studio train, test = ( display(train) from pyspark.ml import Pipeline model = Pipeline( --Updated-- It works if we use Spark 3.3 pools |
We're running into the same issue with this configuration from Azure Synapse Spark 3.4 pool. |
Hi All, thank you for your patience. We have tried to get a simple repro of this and found that its something funky is going on with the natives. ...
.setDataTransferMode("bulk") To work around this for now |
Confirm this works in Synapse Spark 3.4 pool. Thank you @mhamilton723 for the mitigation. |
Can you elaborate how to use the workaround. Thank you! |
@Jens-automl - it's just a function on the LightGBMClassifier https://mmlspark.blob.core.windows.net/docs/1.0.4/pyspark/synapse.ml.lightgbm.html |
from synapse.ml.lightgbm import LightGBMClassifier model = LightGBMClassifier( |
SynapseML version
1.0.4
System information
Runtime 1.2
Describe the problem
I am trying the following tutorial in a Microsoft Fabric notebook:
https://learn.microsoft.com/en-us/fabric/data-science/how-to-use-lightgbm-with-synapseml
The Step 5 of this sample is failing:
Here is the excerpt of the error:
Code to reproduce issue
Other info / logs
No response
What component(s) does this bug affect?
area/cognitive
: Cognitive projectarea/core
: Core projectarea/deep-learning
: DeepLearning projectarea/lightgbm
: Lightgbm projectarea/opencv
: Opencv projectarea/vw
: VW projectarea/website
: Websitearea/build
: Project build systemarea/notebooks
: Samples under notebooks folderarea/docker
: Docker usagearea/models
: models related issueWhat language(s) does this bug affect?
language/scala
: Scala source codelanguage/python
: Pyspark APIslanguage/r
: R APIslanguage/csharp
: .NET APIslanguage/new
: Proposals for new client languagesWhat integration(s) does this bug affect?
integrations/synapse
: Azure Synapse integrationsintegrations/azureml
: Azure ML integrationsintegrations/databricks
: Databricks integrationsThe text was updated successfully, but these errors were encountered: