Skip to main content
cancel
Showing results for 
Search instead for 
Did you mean: 

Register now to learn Fabric in free live sessions led by the best Microsoft experts. From Apr 16 to May 9, in English and Spanish.

Reply
dwpurdy
Regular Visitor

Can you read a Fabric lakehouse view from spark?

I have a Fabric lake house with a view that I created via the T-SQL endpoint.  Can I query the view from a spark notebook?  If so how?

5 REPLIES 5
SorenSparso
Frequent Visitor

Hi,

 

Has anyone found a solution/workaround for this? Due to the nature of the project, I have to use T-SQL, so writing the view definition SQL directly in the notebook wont work.

 

Thanks

puneetvijwani
Resolver IV
Resolver IV

@dwpurdy you should be using below way in your notebook cells to create and use that view logic and use as a source in notebook to write out as delta table 

%%sql
CREATE OR REPLACE TEMP VIEW Viewname AS
select statmenents 


PS:If my response helped, kindly select it as the solution. Your kudos are greatly appreciated!
dwpurdy
Regular Visitor

Thanks for the work around, but I was trying to encapusulate transformation logic in T-SQL views and then use the views as sources to write out new delta tables wth a Spark script. 

 

It's hard to believe it's not possible to read a T-SQL view via the end point in Spark.  Does that also mean, I can't access warehouse tables in Spark via the warehouse SQL end point? 

 

This seems like a missing feature. In Synapse, it's easy to read a dedicated pool table in Spark.

It's not possible to read a T-SQL view from Spark. But it's possible to read a warehouse table using Spark. You just need to add the warehouse table as a shortcut and you can access those tables just like the other lakehouse tables. 

HimanshuS-msft
Community Support
Community Support

Hello @dwpurdy 

I did tried to access the views from a notebook and spark always complained that the object does not exists .

%%sql
SELECT * FROM view1

Error: Table or view not found: view1; line 1 pos 14;
'Project [*]
+- 'UnresolvedRelation [view1], [], false

So i am pretty sure that you cannot access from SPARK  . 

Since views are just SQL query  what you can do as work around is , run the SQL query itself 

 
%%sql
SELECT * FROM Address


Thanks
HImanshu

Helpful resources

Announcements
Microsoft Fabric Learn Together

Microsoft Fabric Learn Together

Covering the world! 9:00-10:30 AM Sydney, 4:00-5:30 PM CET (Paris/Berlin), 7:00-8:30 PM Mexico City

March 2024  FBC Gallery Image

Fabric Monthly Update - March 2024

Check out the March 2024 Fabric update to learn about new features.

April Fabric Community Update

Fabric Community Update - April 2024

Find out what's new and trending in the Fabric Community.

Top Kudoed Authors