Forum Discussion

Mauro89's avatar
Mauro89
Super User
8 months ago
Solved

AI functions with PySpark error

Hi there,   I have tried the AI functions capabilities in notebooks. The example with pandas dataframe worked for me. But the example as of the MS documentation with PySpark does not work. Example...
Notebook
  • Mauro89's avatar
    Mauro89
    8 months ago

    v-karpurapud sooooo. I turned out to be the issue was sitting infront of the screen not in the screen 😅

    I was only importing the libary for pandas and was not award that the pyspark library is a different one. So for all ready this post, the solution is to import the correct relevant library.

     

    import synapse.ml.aifunc as aifunc --> for Pandas
    import synapse.ml.spark.aifunc as aifunc --> for PySpark
     
    Nevertheless, thanks for the support here in this thread!
    v-karpurapud may I can take my own answer as solution?

     

Mauro89's avatar
Mauro89
Super User
8 months ago

Hi v-karpurapud,

 

Iam still not sure to which extension you relate (AI Spark Extensions being enabled on the specific capacity)?
Is it in the admin portal or where can I find it? As Iam Admin in the Fabric environment as well as capacity I assume I can enable it once found.

 

And the workaround throughs the follwoing error:
No module named 'fabricext'

 

Thanks and best regards!

v-karpurapud's avatar
v-karpurapud
Community Support
8 months ago

Hi Mauro89 

 

To clarify, when I mentioned “AI Spark Extensions being enabled on the capacity,” I did not mean there is a manual setting in the Admin Portal, as such an option does not exist. The Spark AI layer is included, but Microsoft activates it automatically during the Fabric Runtime rollout, and its availability may differ based on capacity or region.

In this context, “enabled” signifies that the Spark AI components are present in the capacity’s runtime and accessible to notebooks. In your case, the absence of both df.ai and the fabricext module suggests that the Spark AI package is not yet available on your capacity, despite being on Runtime 1.3 with tenant-level Copilot/OpenAI enabled.

Since this extension is generally expected to be available, we recommend submitting a support ticket so the Fabric team can review your capacity and assist further.

You can submit a ticket via the Microsoft Power BI Support Portal:

How to create a Fabric and Power BI Support ticket - Power BI | Microsoft Learn

 

Regards,

Microsoft Fabric Community Support Team.

  • Mauro89's avatar
    Mauro89
    Super User
    8 months ago

    Thanks v-karpurapud for clarification.

    I have created a ticket and will keep this thread posted about the solution.

    • v-karpurapud's avatar
      v-karpurapud
      Community Support
      8 months ago

      Hi Mauro89 

      Thanks for the update. Hope your issue gets resolved soon. when it does, please share the insights with the community as it can be helpful to others.