Hi all, I was following the guide:Create models with Automated ML (preview)When I reached the step to train a LightGBMClassifier model on the provided training data, I encountered the following warning and errors. May I know if anyone has encountered this and any way to overcome this? "WARNING mlflow.data.spark_dataset: Failed to infer schema for Spark dataset. Exception: Unsupported Spark Type '<class 'pyspark.ml.linalg.VectorUDT'>', MLflow schema is only supported for scalar Spark types." Cell In[104], line 9 6 with mlflow.start_run(run_name="default") as run: 7 # Create a LightGBMClassifier model with specified settings 8 model = LightGBMClassifier(objective="binary", featuresCol="features", labelCol="Exited") ----> 9 model = model.fit(train_data) File ~/cluster-env/trident_env/lib/python3.10/site-packages/mlflow/utils/autologging_utils/safety.py:573, in safe_patch.<locals>.safe_patch_function(*args, **kwargs) 571 patch_function.call(call_original, *args, **kwargs) 572 else: --> 573 patch_function(call_original, *args, **kwargs) 575 session.state = "succeeded" 577 try_log_autologging_event( 578 AutologgingEventLogger.get_logger().log_patch_function_success, 579 session, (...) 583 kwargs, 584 )

Hi DCHTan_SG Apologies for the inconvenience. Please reach out to our support team to gain deeper insights and explore potential solutions. It's highly recommended that you reach out to our support team. Their expertise will be invaluable in suggesting the most appropriate approach. Please go ahead and raise a support ticket to reach our support team: https://support.fabric.microsoft.com/support After creating a Support ticket please provide the ticket number as it would help us to track for more information. Thank you.

MLflow schema is only supported for scalar Spark types

Forum Discussion

Related content

MLFlow - Problem with aliases and metrics

MLFlow - Not logging results of all epochs

Campaign Voucher FAQ & Support

Does Data Science notebooks in R support Experiments / MLFLOW

Eventstream read EventHub with Avro format and Schema Registry (not schema embedded)