Forum Discussion

homanz's avatar
homanz
Icon for Advocate I rankAdvocate I
4 months ago
Solved

Temp views don't work when selecting tables with a schema name using Spark SQL

I created a new temporary view using Spark SQL. Code is below   CREATE OR REPLACE TEMPORARY VIEW tmpvw1 AS SELECT * FROM IM.test   In next step, I tried to select the temp view tmpvw1 using...
data engineering
notebook
  • v-veshwara-msft's avatar
    v-veshwara-msft
    4 months ago

    Hi homanz ,


    Thanks for reaching out to Microsoft Fabric Community.

    At the moment, this appears to be the current behavior of Spark SQL in schema enabled Lakehouses rather than an issue that can be resolved through an immediate hotfix. The PySpark approach you identified is currently the recommended workaround.

     

    Microsoft has also indicated that schema enabled Lakehouses in Spark still have some limitations, with additional support and improvements expected to roll out gradually over the coming months.

    Lakehouse Schemas (Generally Available) - Microsoft Fabric Community

     

    For more details: Spark views in lakehouses - Microsoft Fabric | Microsoft Learn

     

    You may also consider raising this as an enhancement request through the Fabric Ideas - Microsoft Fabric Community so the product team can review and prioritize support for this scenario in future updates.

     

    Hope this helps. Please reach out for further assistance.
    Thank you.

deborshi_nag's avatar
deborshi_nag
Icon for Super User rankSuper User
4 months ago

Hello homanz 

 

Fabric relies on an internal session database for Spark session and lakehouse binding. If you run SELECT current_database() as current_db, you'll notice a temporary database name appears.

 

To resolve this, always use a four-part name with backticks—`<Your Workspace Name>`.Lakehouse.Schema.Table—when referencing tables. You can find your schema's namespace by running the SQL below and checking the "namespace" column.

SHOW TABLES in IM;