Forum Discussion
Problems with the AutoML process
Hi RobertoNavarro,
It seems like I forget the recently update on the spark runtime environments, they may affect all the packages usage that requires the specific environment versions.
For this scenario, you can wait for the new version release or try to contact to the AutoML package dev team to update the compatibility version.
Regards,
Xiaoxin Sheng
Hi,
I think that I have found a possible answer, if I execute help(modules) among all the output, I have got this:
****** AutoML Prerequisites ******
AutoML requires an Apache Spark pool (version 2.4) in your Azure Synapse Analytics workspace.
You are using a Spark 3.x pool, please use Spark 2.x pool. For details, see: https://docs.microsoft.com/en-us/azure/synapse-analytics/machine-learning/tutorial-automl#prerequisites
And in this webpage:
Warning
- Effective September 29, 2023, Azure Synapse will discontinue official support for Spark 2.4 Runtimes. Post September 29, 2023, we will not be addressing any support tickets related to Spark 2.4. There will be no release pipeline in place for bug or security fixes for Spark 2.4. Utilizing Spark 2.4 post the support cutoff date is undertaken at one's own risk. We strongly discourage its continued use due to potential security and functionality concerns.
- As part of the deprecation process for Apache Spark 2.4, we would like to notify you that AutoML in Azure Synapse Analytics will also be deprecated. This includes both the low code interface and the APIs used to create AutoML trials through code.
- Please note that AutoML functionality was exclusively available through the Spark 2.4 runtime.
- For customers who wish to continue leveraging AutoML capabilities, we recommend saving your data into your Azure Data Lake Storage Gen2 (ADLSg2) account. From there, you can seamlessly access the AutoML experience through Azure Machine Learning (AzureML). Further information regarding this workaround is available here.
So I think that this feature of AutoML cannot be use either in fabric regarding that the underlying engine is Synapse Spark Pools, is this correct?
Hi Anonymous,
Any comment about this? I have received a couple of e-mails about if this issue is closed, but I don't have a solution, is it necessary more time to analize the problem from your side?
Regards and thanks for your support
Roberto
Hi RobertoNavarro,
It seems like I forget the recently update on the spark runtime environments, they may affect all the packages usage that requires the specific environment versions.
For this scenario, you can wait for the new version release or try to contact to the AutoML package dev team to update the compatibility version.
Regards,
Xiaoxin Sheng