Forum Discussion
Creating views from spark notebook into SQL analytics endpoint?
You can create the views directly in the Lakehouse's SQL Analytics Endpoint.
You can create the views directly in the Lakehouse's SQL Analytics Endpoint.
Thankyou, I was worried this was the case. I was hoping that I could use spark notebooks because I cant automate the run of view scripts in a lakehouse via a pipeline.
Hi somewhereels ,
If you are looking for a method to query the lakehouse from fabric pipeline then you can achieve a similar result using a script activity.
Take the SQL analytics endpoint from the lakehouse to create a new cloud connection - use External as an option and run the query against the cloud connection within the pipeline. Just ran and works.
Steps to connect to Lakehouse Endpoint using Script Activity:
1. Copy the SQL Endpoint from Lakehouse.
2. In Script, click on External and then Azure SQL Database, fill the details and use Organizational Account to establish connection.
3. Check the connection
4. Execute it
In your case, if you want to create a view,Result:
FYI: Support for SQL endpoint for Lakehouse is in our backlog but we don't have a ETA.
Hope this is is helpful. Please let me know incase of further queries.Hi somewhereels ,
We haven’t heard from you on the last response and was just checking back to see if workaround helped you?Hi yes I can get this solution to work, thankyou for your time. I don't think I tried this because I was under the assumption that the sql endpoint was read-only?
Hello
This helps. I tried to create a view using script activity via the sql endpoint connection and it works.
However I couldn't read this view from notebook and its not even shown in lakehoue view. This is expected?
I tried to create a view from notebook, it fails because it will support only managed tables.
Can you suggest something. ?
